-- Logs begin at Sat 2022-09-17 03:48:15 UTC, end at Sat 2022-09-17 04:58:20 UTC. -- Sep 17 04:13:58.630170 dv-cinder-857993-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 17 04:13:58.639000 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 17 04:13:58.641166 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 17 04:13:58 2022] *** Sep 17 04:13:58.641166 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 17 04:13:58.641166 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 17 04:13:58.641166 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: nodename: dv-cinder-857993-2-iscsi Sep 17 04:13:58.641166 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: machine: x86_64 Sep 17 04:13:58.641166 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: clock source: unix Sep 17 04:13:58.641166 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: pcre jit disabled Sep 17 04:13:58.641166 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: detected number of CPU cores: 4 Sep 17 04:13:58.641166 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: current working directory: / Sep 17 04:13:58.641166 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: detected binary path: /usr/bin/uwsgi-core Sep 17 04:13:58.641715 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: your processes number limit is 47680 Sep 17 04:13:58.641715 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: your memory page size is 4096 bytes Sep 17 04:13:58.641715 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: detected max file descriptor number: 2048 Sep 17 04:13:58.641715 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: lock engine: pthread robust mutexes Sep 17 04:13:58.641715 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: thunder lock: enabled Sep 17 04:13:58.641715 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 17 04:13:58.641715 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 17 04:13:58.680061 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: Python main interpreter initialized at 0x55bae96eb9c0 Sep 17 04:13:58.680061 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: python threads support enabled Sep 17 04:13:58.680061 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: your server socket listen backlog is limited to 100 connections Sep 17 04:13:58.680061 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: your mercy for graceful operations on workers is 90 seconds Sep 17 04:13:58.680633 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: mapped 403077 bytes (393 KB) for 2 cores Sep 17 04:13:58.680633 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: *** Operational MODE: preforking *** Sep 17 04:13:58.680825 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: *** uWSGI is running in multiple interpreter mode *** Sep 17 04:13:58.680825 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: spawned uWSGI master process (pid: 81403) Sep 17 04:13:58.680979 dv-cinder-857993-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 17 04:13:58.681559 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: spawned uWSGI worker 1 (pid: 81417, cores: 1) Sep 17 04:13:58.682027 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: spawned uWSGI worker 2 (pid: 81418, cores: 1) Sep 17 04:13:58.682289 dv-cinder-857993-2-iscsi devstack@keystone.service[81403]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 17 04:14:02.845742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 04:14:02.845742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: from cryptography.utils import int_from_bytes Sep 17 04:14:02.846585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 04:14:02.846585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: from cryptography.utils import int_from_bytes Sep 17 04:14:02.847002 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 04:14:02.847002 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: from cryptography.utils import int_from_bytes Sep 17 04:14:02.847111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 04:14:02.847111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: from cryptography.utils import int_from_bytes Sep 17 04:14:02.895268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 04:14:02.895468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 04:14:02.895601 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] command line args: [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 04:14:02.895694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 04:14:02.895880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 04:14:02.896014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 04:14:02.896014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ================================================================================ {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 04:14:02.896150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] command line args: [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 04:14:02.896281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] admin_token = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.896366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 04:14:02.896476 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ================================================================================ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 04:14:02.896564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] config_dir = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.896672 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.896754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] admin_token = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.896813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] config_source = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.896926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] config_dir = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.897002 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] debug = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.897149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.897241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.898368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] config_source = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.898368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] debug = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.898494 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.898564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.898653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.898718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.898776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.898833 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.898899 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.898962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_config_append = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899314 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_dir = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_config_append = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899432 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_file = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_dir = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_file = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899903 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.899966 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.900141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.900211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.900276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.900339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.900524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.900712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.900712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.900912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.901036 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.901157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.901296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.901416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.901500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.901578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.901708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.901788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.901868 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.901990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.902065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.902182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.902346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.902470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.902565 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.902832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.902967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.903110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] publish_errors = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.903202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.903280 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.903435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.903558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] publish_errors = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.903650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.903732 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.903810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.903944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.904027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.904166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.904290 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] standard_threads = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.904368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.904439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.904509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.904699 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.904832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] standard_threads = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.904912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.904991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.905071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] use_journal = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.905183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.905314 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] use_json = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.905389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.905457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] use_stderr = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.905534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] use_syslog = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.905606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_journal = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.905675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_json = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.905760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.905832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_stderr = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.905974 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.906097 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_syslog = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.906176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.906247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:14:02.906448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.906571 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.906660 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.906732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.906913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.907034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.907126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.907200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.907376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.907497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.907574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.907645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.907838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.907971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.908052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.908129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.908297 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.908430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.908519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.908598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.908760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.908883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.908960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.909030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.909242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.909371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.909449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.909515 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.909703 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.909824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.909916 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.909990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.910166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.910249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.910399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.910478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.910641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.910721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.910863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.910944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.911111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.911200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.911353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.911454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.911595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.911676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.911820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.911909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.912042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.912123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.912267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.912349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.912500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.912590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.912728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.912808 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.912963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.913044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.913222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.913321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.913449 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.913530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.913681 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.913761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.913908 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.914000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.914122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.914205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.914364 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.914440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.914603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.914775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.914870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.914948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.915097 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.915178 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.915323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.915407 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.915549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.915633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.915776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.915852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.916001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.916153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.916235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.916304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.916442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.916519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.916677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.916766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.916908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.916985 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.917156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.917239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.917390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.917485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.917620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.917697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.917859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.917941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.918083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.918166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.918357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.918478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.918577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.918669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.918762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.918863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.918983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.919085 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.919217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.919306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.919425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.919513 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.919654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.919762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.919878 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.919991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.920100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.920216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.920329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.920439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.920550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.920663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.920782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.920903 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.921027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.921182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.921288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.921398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.921509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.921627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.921744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.921855 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.921967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.922068 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.922177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.922287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.922405 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.922517 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.922628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.922739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.922846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.922964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.923084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.923196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.923307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.923417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.923528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.923638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.923758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.923871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.923987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.924099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.924210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.924320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.924438 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.924550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.924661 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.924771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.924882 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.924992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.925165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.925289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.925401 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.925512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.925629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.925740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.925861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.925974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.926084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.926201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.926313 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.926423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.926546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.926660 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.926770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.926881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.926992 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.927109 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.927298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.927416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.927528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.927639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.927751 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.927880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.928004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.928125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.928246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.928365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.928478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.928591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.928703 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.928822 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.928941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.929063 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.929191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.929305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.929416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.929539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.929653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.929765 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.929876 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.929987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.930099 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.930218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.930338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.930452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.930571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.930684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.930804 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.external = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.930951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.931074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.931202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.931323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.931438 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.external = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.931550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.931670 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.931784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.password = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.931904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.932018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.token = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.932132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.932244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.932355 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.password = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.932466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.932587 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.token = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.932701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.932812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.932924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.933035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.933200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.933327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.933443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.933556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.933668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.933780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.933892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.934013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.934127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.934240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.934343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.934454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.934574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.934697 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.934812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.934924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.935043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.935157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.935270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.935394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.935509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.935621 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.935734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.935846 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.935964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.936087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.936201 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.936314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.936426 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.936539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.936651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.936772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.936887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.936999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.937126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.937250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.937366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.937486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.937607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.937720 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.937843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.937843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.938055 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.938055 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.938276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.938276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.938487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.938606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.938731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.938845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.938957 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.939069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.939182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.939300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.939421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.939537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.939650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.939761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.939880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.940091 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.940224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.940339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.940454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.940567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.940688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.940806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.940999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.941148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.941264 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.941376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.941488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.941602 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.941731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.941846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.941959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.942071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.942183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.942295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.942415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.942538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.942653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.942767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.942879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.942991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.943112 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.943226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.943338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.943450 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.943562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.943676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.943796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.943910 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.944023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.944136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.944248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.944361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.944481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.944596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.944717 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.944830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.944943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.945056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.945200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.945315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.945428 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.945543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.945654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.945768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.945940 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.946060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.946173 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.946286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.946398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.946510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.946631 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.946816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.946933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.947045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.947157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.947269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.947389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.947503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.947615 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.947727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.947842 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.947956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.948076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.948191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.948302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.948414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.948525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.948646 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.948760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.948873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.949074 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.949240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.949352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.949477 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.949591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.949704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.949816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.949935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.950049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.950170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.950283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.950396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.950507 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.950619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.950730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.950851 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.950965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.951078 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.951189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.951301 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.951412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.951532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.951645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.951757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.951869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.951980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.952093 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.952219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.952333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.952446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.952557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.952669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.952781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.952900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.953014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.953140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.953254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.953366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.953477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.953600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.953715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.953828 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.953939 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.954050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.954162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.954282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.954396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.954508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.954508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.954718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.954830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.954949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.955062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.955174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.955287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.955399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.955399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.955617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.955730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.955842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.955953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.956065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.956175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.956296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.956410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.956522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.956634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.956745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.956745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.956963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.957093 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.957209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.957322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.957433 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.957544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.957669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.957783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.957898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.958010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.958121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.958232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.958353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.958467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.958578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.958690 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.958801 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.958912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.959031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.959144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.959262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.959374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.959486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.959597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.959717 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.959837 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.959950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.960061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.960173 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.960284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.960404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.960524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.960637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.960749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.960860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.960971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.961145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.961261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.961373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.961485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.961597 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.961708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.961832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.961946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.962058 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.962170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.962281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.962393 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.962513 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.962628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.962739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.962850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.962961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.963151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] role.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.963266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.963379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] role.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.963491 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.963604 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.963715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.963836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.963949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.964062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.964175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] role.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.964287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.964398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] role.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.964522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.964638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.964749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.964862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.964974 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.965104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.965225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.965324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.965417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.965510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.965604 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.965699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.965801 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.965898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.965993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.966087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.966181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.966275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.966378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.966474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.966474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.966649 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.966765 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.966878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.966998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.967113 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.967225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.967337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.967337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.967547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.967547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.967547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.967547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.967547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.968065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.968184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.968299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.968413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.968524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.968637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.968749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.968870 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.968984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.969125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.969240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.969352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.969464 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.969588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.969703 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.969815 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.969927 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.970039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.970157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.970278 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.970392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.970503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.970615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.970727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.970839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.970959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.971072 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.971187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.971298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.971409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.971521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.971640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.971754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.971866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.971978 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.972090 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.972202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.972322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.972426 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.972521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.972614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.972725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.972838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.972959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.973074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.973206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.973318 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.973430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.973541 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.973665 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.973779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.973891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.974003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.974116 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.974228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.974348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.974462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.974573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.974685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.974805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.974932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.975054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.975170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.975282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.975403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.connection = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.975516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.975629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.975741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.975844 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.975954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.976080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.976196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.976308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.976420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.976533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.976644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.976766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.976880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.976993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.977217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.977336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.977450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.977576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.977692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.977805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.977920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.978066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.978180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.978303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.978418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.978532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.978644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.978757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.978869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.978989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.979104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.979217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.979330 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.979442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.979554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.979675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.979675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 04:14:02.979889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:14:02.979889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 04:14:03.008042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.008263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.008367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.008543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.008626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.008782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.016672 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.016978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.044187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.044411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.044772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.044966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.096344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.096598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.144206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.144346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.144478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.144584 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.301806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.302024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.302202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.302414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.302641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.302847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.304686 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.304919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.305143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.305424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.305643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.305858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.325204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.325378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.325512 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.325659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.325745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.325814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.325887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.326001 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.326104 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.326217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.326297 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.326408 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.326498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.326646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.337360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.337776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.349737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.349962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.350088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.350249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.350333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.350396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.350463 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.350535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.350611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.350813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.350938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.351076 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.351158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.351158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 04:14:03.367422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.368034 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 04:14:03.368379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.368505 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.368644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 04:14:03.368747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 04:14:03.368865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.368946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.369041 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.369164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 04:14:03.369345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.369439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.369616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.369743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.369881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.370089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.379065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.379340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.379607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.379733 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.379872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.379974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.380163 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.380445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.386810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.387661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.389691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.389975 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.390254 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.390530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.390656 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.390797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.390898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.391016 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.391097 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.391303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.391431 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.391584 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.391668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.391826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.391954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.392092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.392187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.392300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.392381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.392653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.392926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.393221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.414784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.415733 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.416221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.416509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.417220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.417504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.421219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.421500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.421777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.422212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.422494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.422771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.428110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.428395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.429152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.429458 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.434008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.434298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.434580 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.434864 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.435091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.435239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.435379 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.435460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.435612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.435701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.435899 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.436027 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.436163 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.436443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.436713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.436975 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.451215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.451467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.452253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.452531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.455219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.456323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.458396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.458685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.459509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.459800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.462866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.463968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.465913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.467041 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.469040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.469361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.469639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.469914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.470184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.470310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.470465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.470550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.470750 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.471034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.471308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.471585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.485787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.486071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.486335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.486599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.487014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.487315 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.487587 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.487826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.493692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.495186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.496661 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.496903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.497306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.498831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.499556 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.499847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.500123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.500325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.500451 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.500652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.500850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.503435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.503719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.504002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.504283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.504681 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.512122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.512418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.512694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.512969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.513364 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.513853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.514131 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.515796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.516080 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.516360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.516646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.517023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.517352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.517595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.530327 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.533241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.533897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.536076 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.536890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.537595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.539035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.539323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.539774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.539900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.541468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.543031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.543332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.543723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.546170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.547593 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.550469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.550853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.551134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.551411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.558262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.559158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.559448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.559681 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.559808 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.566500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.566629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.566768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.566868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.567082 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.567223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.567361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.567442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.567572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.567847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.568118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.568392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.568662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.568939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.569233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.569491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.569731 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.569987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.577852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.578152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.578441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.585344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 04:14:03.585633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.585917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.586193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.586515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.586801 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.587081 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.587358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.587628 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.587921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.588177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.588439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.588703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.588967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 04:14:03.603884 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bae96eb9c0 pid: 81418 (default app) Sep 17 04:14:03.607221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5b108-c354-4cd7-bf48-53c72d4a06a0 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:03.607504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5b108-c354-4cd7-bf48-53c72d4a06a0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:03.607768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5b108-c354-4cd7-bf48-53c72d4a06a0 None None] PATH_INFO: `/v3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:03.611926 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1/1] 10.222.0.60 () {50 vars in 791 bytes} [Sat Sep 17 04:14:03 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:03.623462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bae96eb9c0 pid: 81417 (default app) Sep 17 04:14:10.364650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7a643-2c72-47bc-a602-2c73d5441933 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:10.365742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7a643-2c72-47bc-a602-2c73d5441933 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:10.365742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7a643-2c72-47bc-a602-2c73d5441933 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:10.368046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 2/2] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:14:10.378302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72f394-b69a-4ec9-9f50-ee2d48f2c914 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:10.378773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72f394-b69a-4ec9-9f50-ee2d48f2c914 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:10.378916 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72f394-b69a-4ec9-9f50-ee2d48f2c914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:10.448411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: INFO dbcounter [None req-0b72f394-b69a-4ec9-9f50-ee2d48f2c914 None None] Registered counter for database keystone Sep 17 04:14:10.469345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG oslo_db.sqlalchemy.engines [None req-0b72f394-b69a-4ec9-9f50-ee2d48f2c914 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81417) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:14:10.477144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writer thread running {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:14:10.656289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG passlib.handlers.bcrypt [None req-0b72f394-b69a-4ec9-9f50-ee2d48f2c914 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81417) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 17 04:14:10.656582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG passlib.handlers.bcrypt [None req-0b72f394-b69a-4ec9-9f50-ee2d48f2c914 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81417) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 17 04:14:10.714713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0b72f394-b69a-4ec9-9f50-ee2d48f2c914 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:10.716313 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0b72f394-b69a-4ec9-9f50-ee2d48f2c914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:10.849541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1/3] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 17 04:14:11.383943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7a643-2c72-47bc-a602-2c73d5441933 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:11.460502 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: INFO dbcounter [None req-37d7a643-2c72-47bc-a602-2c73d5441933 None None] Registered counter for database keystone Sep 17 04:14:11.485400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG oslo_db.sqlalchemy.engines [None req-37d7a643-2c72-47bc-a602-2c73d5441933 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81418) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:14:11.496670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writer thread running {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:14:11.615551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7a643-2c72-47bc-a602-2c73d5441933 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:11.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e7476-6c0b-441c-8139-1d25eeb062a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:11.623266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e7476-6c0b-441c-8139-1d25eeb062a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:11.623545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e7476-6c0b-441c-8139-1d25eeb062a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:11.623814 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e7476-6c0b-441c-8139-1d25eeb062a9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:11.633362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e7476-6c0b-441c-8139-1d25eeb062a9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:11.684422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e13e7476-6c0b-441c-8139-1d25eeb062a9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 17 04:14:11.684761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e7476-6c0b-441c-8139-1d25eeb062a9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:11.695170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 3/4] 10.222.0.60 () {58 vars in 1324 bytes} [Sat Sep 17 04:14:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:14:11.702124 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72f394-b69a-4ec9-9f50-ee2d48f2c914 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:11.717284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72f394-b69a-4ec9-9f50-ee2d48f2c914 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:11.722700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4959d-3fee-439b-b7d1-c081d4ef0e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:11.723834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4959d-3fee-439b-b7d1-c081d4ef0e1d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:11.724117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4959d-3fee-439b-b7d1-c081d4ef0e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:11.724389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4959d-3fee-439b-b7d1-c081d4ef0e1d admin admin] PATH_INFO: `/v3/projects/7fd2fb11102f467d836ef30d6efdd922` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:11.725872 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4959d-3fee-439b-b7d1-c081d4ef0e1d admin admin] RBAC: Authorizing `identity:get_project(project_id=7fd2fb11102f467d836ef30d6efdd922)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:11.778302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c0e4959d-3fee-439b-b7d1-c081d4ef0e1d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 17 04:14:11.778650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4959d-3fee-439b-b7d1-c081d4ef0e1d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:11.783196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 2/5] 10.222.0.60 () {56 vars in 1207 bytes} [Sat Sep 17 04:14:11 2022] GET /identity/v3/projects/7fd2fb11102f467d836ef30d6efdd922 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:13.923093 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf70a0-6e3c-4876-a042-2c44f4bf5577 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:13.923968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf70a0-6e3c-4876-a042-2c44f4bf5577 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:13.923968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf70a0-6e3c-4876-a042-2c44f4bf5577 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:13.930571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 4/6] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:13.936313 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575b9cc-52dc-4de7-baf7-687876b1e7f2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:13.936751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575b9cc-52dc-4de7-baf7-687876b1e7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:13.936842 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575b9cc-52dc-4de7-baf7-687876b1e7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:13.959352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6575b9cc-52dc-4de7-baf7-687876b1e7f2 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:13.960859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6575b9cc-52dc-4de7-baf7-687876b1e7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:13.970948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 3/7] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 17 04:14:14.264141 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf70a0-6e3c-4876-a042-2c44f4bf5577 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:14.286849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf70a0-6e3c-4876-a042-2c44f4bf5577 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:14.294159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54debc27-a5dc-4664-b552-14b883589018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:14.294761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54debc27-a5dc-4664-b552-14b883589018 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:14.295039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54debc27-a5dc-4664-b552-14b883589018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:14.295310 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54debc27-a5dc-4664-b552-14b883589018 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:14.303304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54debc27-a5dc-4664-b552-14b883589018 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:14.305066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54debc27-a5dc-4664-b552-14b883589018 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:14.313876 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 5/8] 10.222.0.60 () {58 vars in 1324 bytes} [Sat Sep 17 04:14:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:14:14.319667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575b9cc-52dc-4de7-baf7-687876b1e7f2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:14.329232 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575b9cc-52dc-4de7-baf7-687876b1e7f2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:14.333872 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bafac-4384-49fa-9988-16665191af81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:14.334520 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bafac-4384-49fa-9988-16665191af81 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:14.334793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bafac-4384-49fa-9988-16665191af81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:14.335058 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bafac-4384-49fa-9988-16665191af81 admin admin] PATH_INFO: `/v3/users/8a7b708ff6c140a3ba833eb00eff7926` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:14.336427 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160bafac-4384-49fa-9988-16665191af81 admin admin] RBAC: Authorizing `identity:get_user(user_id=8a7b708ff6c140a3ba833eb00eff7926)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:14.338464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160bafac-4384-49fa-9988-16665191af81 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:14.340342 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 4/9] 10.222.0.60 () {56 vars in 1198 bytes} [Sat Sep 17 04:14:14 2022] GET /identity/v3/users/8a7b708ff6c140a3ba833eb00eff7926 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:16.570562 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525c0f8-6f34-44a0-b987-67d612555d3c None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:16.571142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525c0f8-6f34-44a0-b987-67d612555d3c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:16.571142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525c0f8-6f34-44a0-b987-67d612555d3c None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:16.572445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 6/10] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:16.577814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc4274-59fb-43cd-8e14-07446515034b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:16.578246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc4274-59fb-43cd-8e14-07446515034b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:16.578339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc4274-59fb-43cd-8e14-07446515034b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:16.596957 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-88fc4274-59fb-43cd-8e14-07446515034b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:16.598187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-88fc4274-59fb-43cd-8e14-07446515034b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:16.623957 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994782a8-a9b5-43c5-babf-3bc80d1b53bb None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:16.624239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994782a8-a9b5-43c5-babf-3bc80d1b53bb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:16.624510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994782a8-a9b5-43c5-babf-3bc80d1b53bb None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:16.625876 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 7/11] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:16.631093 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d9e24-03a1-4530-b40f-c372dd64bf79 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:16.631377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d9e24-03a1-4530-b40f-c372dd64bf79 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:16.631711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d9e24-03a1-4530-b40f-c372dd64bf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:16.674275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG passlib.handlers.bcrypt [None req-536d9e24-03a1-4530-b40f-c372dd64bf79 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81418) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 17 04:14:16.674668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG passlib.handlers.bcrypt [None req-536d9e24-03a1-4530-b40f-c372dd64bf79 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81418) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 17 04:14:16.679460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 5/12] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:16.723343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-536d9e24-03a1-4530-b40f-c372dd64bf79 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:16.724921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-536d9e24-03a1-4530-b40f-c372dd64bf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:16.825962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 8/13] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:16.941338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc4274-59fb-43cd-8e14-07446515034b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:16.965577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:17.046028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc4274-59fb-43cd-8e14-07446515034b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:17.082052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42728b-e00e-4cae-b1da-0e4cafffc7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:17.082594 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42728b-e00e-4cae-b1da-0e4cafffc7ac None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:17.082870 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42728b-e00e-4cae-b1da-0e4cafffc7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:17.083141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42728b-e00e-4cae-b1da-0e4cafffc7ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:17.091019 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d9e24-03a1-4530-b40f-c372dd64bf79 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:17.148988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42728b-e00e-4cae-b1da-0e4cafffc7ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:17.150524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42728b-e00e-4cae-b1da-0e4cafffc7ac None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:17.173275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:17.194003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d9e24-03a1-4530-b40f-c372dd64bf79 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:17.196306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:17.230567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6601e-39ca-475b-a95a-0eda4a24b5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:17.231688 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6601e-39ca-475b-a95a-0eda4a24b5ee None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:17.231971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6601e-39ca-475b-a95a-0eda4a24b5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:17.232242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6601e-39ca-475b-a95a-0eda4a24b5ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:17.244084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6601e-39ca-475b-a95a-0eda4a24b5ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:17.247394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 6/14] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:17.252988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42728b-e00e-4cae-b1da-0e4cafffc7ac None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e1a6601e-39ca-475b-a95a-0eda4a24b5ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:17.253325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:17.254218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:17.255051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 9/15] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:14:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:14:17.260236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6601e-39ca-475b-a95a-0eda4a24b5ee None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:17.346096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42728b-e00e-4cae-b1da-0e4cafffc7ac None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:17.357730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6601e-39ca-475b-a95a-0eda4a24b5ee None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:17.369247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:17.380943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77794753-7adf-4d17-8160-6c76b27d34ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:17.381630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77794753-7adf-4d17-8160-6c76b27d34ab None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:17.381911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77794753-7adf-4d17-8160-6c76b27d34ab None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:17.382180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77794753-7adf-4d17-8160-6c76b27d34ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:17.391163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e8b4a-ca8a-4a13-930f-921efe2167d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:17.391764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e8b4a-ca8a-4a13-930f-921efe2167d1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:17.392035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e8b4a-ca8a-4a13-930f-921efe2167d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:17.392301 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e8b4a-ca8a-4a13-930f-921efe2167d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:17.392404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77794753-7adf-4d17-8160-6c76b27d34ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:17.392955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e8b4a-ca8a-4a13-930f-921efe2167d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:17.393287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e8b4a-ca8a-4a13-930f-921efe2167d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:17.393906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77794753-7adf-4d17-8160-6c76b27d34ab None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:17.394703 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e8b4a-ca8a-4a13-930f-921efe2167d1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:17.406243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 10/16] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:17.411874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e8b4a-ca8a-4a13-930f-921efe2167d1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-77794753-7adf-4d17-8160-6c76b27d34ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:14:17.411988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:17.412971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:14:17.414014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:14:17.414014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:14:17.414014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:14:17.414014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 7/17] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:14:17.418682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77794753-7adf-4d17-8160-6c76b27d34ab None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:17.442696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:17.515699 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e8b4a-ca8a-4a13-930f-921efe2167d1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:17.515885 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77794753-7adf-4d17-8160-6c76b27d34ab None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:17.548313 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4bdcb-1de7-493a-848d-4fc8853ad916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:17.548492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cd81c-6e46-44ee-ab7d-7fabf5794eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:17.548915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4bdcb-1de7-493a-848d-4fc8853ad916 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:17.549046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cd81c-6e46-44ee-ab7d-7fabf5794eca None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:17.549174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4bdcb-1de7-493a-848d-4fc8853ad916 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:17.549319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cd81c-6e46-44ee-ab7d-7fabf5794eca None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:17.549456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4bdcb-1de7-493a-848d-4fc8853ad916 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:17.549557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cd81c-6e46-44ee-ab7d-7fabf5794eca None admin] PATH_INFO: `/v3/users/8a7b708ff6c140a3ba833eb00eff7926` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:17.550103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4bdcb-1de7-493a-848d-4fc8853ad916 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:17.550403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4bdcb-1de7-493a-848d-4fc8853ad916 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:17.551100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591cd81c-6e46-44ee-ab7d-7fabf5794eca None admin] RBAC: Authorizing `identity:get_user(user_id=8a7b708ff6c140a3ba833eb00eff7926)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:17.551779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4bdcb-1de7-493a-848d-4fc8853ad916 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:17.552535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591cd81c-6e46-44ee-ab7d-7fabf5794eca None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:17.554147 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 11/18] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:17 2022] GET /identity/v3/users/8a7b708ff6c140a3ba833eb00eff7926 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:17.559736 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cd81c-6e46-44ee-ab7d-7fabf5794eca None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:17.569680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 8/19] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:17.661455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:17.671834 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cd81c-6e46-44ee-ab7d-7fabf5794eca None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:17.709113 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d66a1-9208-499c-84e8-17dead5efb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:17.709774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d66a1-9208-499c-84e8-17dead5efb14 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:17.710074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d66a1-9208-499c-84e8-17dead5efb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:17.710349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d66a1-9208-499c-84e8-17dead5efb14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:17.711575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d66a1-9208-499c-84e8-17dead5efb14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:17.713123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d66a1-9208-499c-84e8-17dead5efb14 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:17.714767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 12/20] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:17.721019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4bdcb-1de7-493a-848d-4fc8853ad916 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:17.760066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:17.827085 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4bdcb-1de7-493a-848d-4fc8853ad916 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:17.862726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842f957-a038-48ad-86cc-3509bbee8c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:17.863351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842f957-a038-48ad-86cc-3509bbee8c76 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:17.863677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842f957-a038-48ad-86cc-3509bbee8c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:17.863954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842f957-a038-48ad-86cc-3509bbee8c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:17.864689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842f957-a038-48ad-86cc-3509bbee8c76 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.domain.id=default, user.id=8a7b708ff6c140a3ba833eb00eff7926) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:17.864996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842f957-a038-48ad-86cc-3509bbee8c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:17.866545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842f957-a038-48ad-86cc-3509bbee8c76 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:17.874120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 9/21] 10.222.0.60 () {56 vars in 1322 bytes} [Sat Sep 17 04:14:17 2022] GET /identity/v3/role_assignments?user.id=8a7b708ff6c140a3ba833eb00eff7926&scope.domain.id=default&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:20.066545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5b1c1-1081-4c01-9339-68cd80500fc3 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:20.067576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5b1c1-1081-4c01-9339-68cd80500fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:20.068075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5b1c1-1081-4c01-9339-68cd80500fc3 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:20.069985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 13/22] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:20.075122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c7194-05c4-48c4-8699-bdac5fda7f0a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:20.075928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c7194-05c4-48c4-8699-bdac5fda7f0a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:20.076396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c7194-05c4-48c4-8699-bdac5fda7f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:20.102095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-057c7194-05c4-48c4-8699-bdac5fda7f0a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:20.103707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-057c7194-05c4-48c4-8699-bdac5fda7f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:20.188356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 10/23] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:20.427935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7298fe-5076-4d27-96a6-a6534b67a124 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:20.428575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7298fe-5076-4d27-96a6-a6534b67a124 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:20.429103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7298fe-5076-4d27-96a6-a6534b67a124 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:20.430638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 14/24] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:20.436050 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8da6e0-ab62-4477-b7e3-50e973e114d9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:20.436570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8da6e0-ab62-4477-b7e3-50e973e114d9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:20.437122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8da6e0-ab62-4477-b7e3-50e973e114d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:20.458885 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6f8da6e0-ab62-4477-b7e3-50e973e114d9 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:20.460358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6f8da6e0-ab62-4477-b7e3-50e973e114d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:20.487297 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:20.547593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 11/25] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:20.621464 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7298fe-5076-4d27-96a6-a6534b67a124 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:20.759051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7298fe-5076-4d27-96a6-a6534b67a124 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:20.805935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef54d9e-1123-4bca-b1fb-dada0b109254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:20.806608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef54d9e-1123-4bca-b1fb-dada0b109254 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:20.806889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef54d9e-1123-4bca-b1fb-dada0b109254 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:20.807163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef54d9e-1123-4bca-b1fb-dada0b109254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:20.830968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef54d9e-1123-4bca-b1fb-dada0b109254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:20.832457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef54d9e-1123-4bca-b1fb-dada0b109254 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-3ef54d9e-1123-4bca-b1fb-dada0b109254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:14:20.848766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:20.849691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:14:20.850486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:14:20.850486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:14:20.850486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:20.850486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 15/26] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:20.856822 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8da6e0-ab62-4477-b7e3-50e973e114d9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:20.869283 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef54d9e-1123-4bca-b1fb-dada0b109254 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:20.896649 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:20.965776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8da6e0-ab62-4477-b7e3-50e973e114d9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:21.016196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef54d9e-1123-4bca-b1fb-dada0b109254 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:21.019271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa28eba-de86-4a3e-b28c-b785f2092619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:21.019906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa28eba-de86-4a3e-b28c-b785f2092619 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:21.020187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa28eba-de86-4a3e-b28c-b785f2092619 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:21.020460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa28eba-de86-4a3e-b28c-b785f2092619 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:21.021143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa28eba-de86-4a3e-b28c-b785f2092619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:21.021450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa28eba-de86-4a3e-b28c-b785f2092619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:21.022863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa28eba-de86-4a3e-b28c-b785f2092619 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:21.046269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 12/27] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:21.052268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be72464-5170-4e10-b4a2-8f3a7b216440 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:21.055298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600da54-a739-4834-960b-03367d87bf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:21.056176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600da54-a739-4834-960b-03367d87bf8b None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:21.056618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be72464-5170-4e10-b4a2-8f3a7b216440 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:21.056904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be72464-5170-4e10-b4a2-8f3a7b216440 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:21.057244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600da54-a739-4834-960b-03367d87bf8b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:21.057708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600da54-a739-4834-960b-03367d87bf8b None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:21.058587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600da54-a739-4834-960b-03367d87bf8b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:21.058766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 13/28] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:21 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:21.059638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be72464-5170-4e10-b4a2-8f3a7b216440 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:21.060716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600da54-a739-4834-960b-03367d87bf8b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:21.093541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:21.175299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be72464-5170-4e10-b4a2-8f3a7b216440 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:21.211082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61fb11-aa5d-47a7-8db6-712790756589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:21.211715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61fb11-aa5d-47a7-8db6-712790756589 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:21.211995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61fb11-aa5d-47a7-8db6-712790756589 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:21.212272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61fb11-aa5d-47a7-8db6-712790756589 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:21.212985 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61fb11-aa5d-47a7-8db6-712790756589 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:21.214509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61fb11-aa5d-47a7-8db6-712790756589 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:21.283894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 16/29] 10.222.0.60 () {60 vars in 1131 bytes} [Sat Sep 17 04:14:20 2022] POST /identity/v3/roles => generated 219 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:14:21.285507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98382667-fd7d-41a5-a22f-ac1c1f2da7e7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:21.285802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98382667-fd7d-41a5-a22f-ac1c1f2da7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:21.286077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98382667-fd7d-41a5-a22f-ac1c1f2da7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:21.306185 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-98382667-fd7d-41a5-a22f-ac1c1f2da7e7 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:21.307767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-98382667-fd7d-41a5-a22f-ac1c1f2da7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:21.360763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 14/30] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:14:21 2022] POST /identity/v3/projects => generated 301 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:14:21.362348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be2d4f-e88d-4781-a481-2678e13db96c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:21.362688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be2d4f-e88d-4781-a481-2678e13db96c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:21.362967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be2d4f-e88d-4781-a481-2678e13db96c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:21.364495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 15/31] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:21.371041 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dfb27-58cb-40d9-bc02-3e46ceb508d2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:21.371331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dfb27-58cb-40d9-bc02-3e46ceb508d2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:21.371654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dfb27-58cb-40d9-bc02-3e46ceb508d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:21.390238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 17/32] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:21.394268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-645dfb27-58cb-40d9-bc02-3e46ceb508d2 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:21.395936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-645dfb27-58cb-40d9-bc02-3e46ceb508d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:21.492043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:21.495722 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 16/33] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:14:21.720722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda2e5b-ad18-4a07-bfe8-05ee879ae1e3 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:21.721071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda2e5b-ad18-4a07-bfe8-05ee879ae1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:21.721311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda2e5b-ad18-4a07-bfe8-05ee879ae1e3 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:21.722894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 18/34] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:21.728742 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06483a6e-6c10-42ad-add9-e951d90a7a9a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:21.729053 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06483a6e-6c10-42ad-add9-e951d90a7a9a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:21.729333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06483a6e-6c10-42ad-add9-e951d90a7a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:21.747906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda2e5b-ad18-4a07-bfe8-05ee879ae1e3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:21.751845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:21.752503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-06483a6e-6c10-42ad-add9-e951d90a7a9a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:21.753580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-06483a6e-6c10-42ad-add9-e951d90a7a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:21.851141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 17/35] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:14:21.852346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd126b-514e-4377-857c-2e08349aeb21 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:21.852638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd126b-514e-4377-857c-2e08349aeb21 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:21.852964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd126b-514e-4377-857c-2e08349aeb21 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:21.859518 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 18/36] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:21.865433 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ed486-9a77-4be0-af2b-240021c2ce99 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:21.865721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ed486-9a77-4be0-af2b-240021c2ce99 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:21.865999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ed486-9a77-4be0-af2b-240021c2ce99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:21.889235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d23ed486-9a77-4be0-af2b-240021c2ce99 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:21.890431 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d23ed486-9a77-4be0-af2b-240021c2ce99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:21.898817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda2e5b-ad18-4a07-bfe8-05ee879ae1e3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:21.934953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb3236-8efe-4d51-9fc3-1f0075e57d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:21.935950 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb3236-8efe-4d51-9fc3-1f0075e57d67 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:21.936606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb3236-8efe-4d51-9fc3-1f0075e57d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:21.936945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb3236-8efe-4d51-9fc3-1f0075e57d67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:21.947887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb3236-8efe-4d51-9fc3-1f0075e57d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-fdfb3236-8efe-4d51-9fc3-1f0075e57d67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:21.953305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:21.954148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:21.954894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 19/37] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:14:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:21.956268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb3236-8efe-4d51-9fc3-1f0075e57d67 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:21.964862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 19/38] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:14:21.970253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ed486-9a77-4be0-af2b-240021c2ce99 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:22.023026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:22.046518 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:22.072242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb3236-8efe-4d51-9fc3-1f0075e57d67 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:22.092245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ed486-9a77-4be0-af2b-240021c2ce99 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:22.114218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce071008-3857-4392-b8f4-16cf24ae5280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:22.114846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce071008-3857-4392-b8f4-16cf24ae5280 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:22.115353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce071008-3857-4392-b8f4-16cf24ae5280 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:22.115775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce071008-3857-4392-b8f4-16cf24ae5280 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:22.116451 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce071008-3857-4392-b8f4-16cf24ae5280 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:22.119191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce071008-3857-4392-b8f4-16cf24ae5280 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:22.128649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14defd-5c73-4954-8ba9-f526443c22e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:22.129399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14defd-5c73-4954-8ba9-f526443c22e9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:22.129676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14defd-5c73-4954-8ba9-f526443c22e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:22.129944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14defd-5c73-4954-8ba9-f526443c22e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:22.130616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14defd-5c73-4954-8ba9-f526443c22e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:22.130907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14defd-5c73-4954-8ba9-f526443c22e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:22.132601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14defd-5c73-4954-8ba9-f526443c22e9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:22.137452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 20/39] 10.222.0.60 () {60 vars in 1131 bytes} [Sat Sep 17 04:14:21 2022] POST /identity/v3/roles => generated 225 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:14:22.138291 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce071008-3857-4392-b8f4-16cf24ae5280 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:22.264286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce071008-3857-4392-b8f4-16cf24ae5280 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:22.269869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 20/40] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:22.271047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14defd-5c73-4954-8ba9-f526443c22e9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:22.294826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:22.308391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0685fb-a9ed-4326-8a96-f4c8727fabbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:22.309057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0685fb-a9ed-4326-8a96-f4c8727fabbd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:22.309367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0685fb-a9ed-4326-8a96-f4c8727fabbd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:22.309645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0685fb-a9ed-4326-8a96-f4c8727fabbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:22.310818 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0685fb-a9ed-4326-8a96-f4c8727fabbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:22.312282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0685fb-a9ed-4326-8a96-f4c8727fabbd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:22.317231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 21/41] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:22.317231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0685fb-a9ed-4326-8a96-f4c8727fabbd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:22.399771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14defd-5c73-4954-8ba9-f526443c22e9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:22.421745 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:22.423083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0685fb-a9ed-4326-8a96-f4c8727fabbd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:22.434037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b276cf4-a581-46e1-88a1-e2ebb1be1413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:22.434676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b276cf4-a581-46e1-88a1-e2ebb1be1413 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:22.434952 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b276cf4-a581-46e1-88a1-e2ebb1be1413 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:22.435228 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b276cf4-a581-46e1-88a1-e2ebb1be1413 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:22.435924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b276cf4-a581-46e1-88a1-e2ebb1be1413 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:22.437322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b276cf4-a581-46e1-88a1-e2ebb1be1413 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:22.452626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 21/42] 10.222.0.60 () {60 vars in 1131 bytes} [Sat Sep 17 04:14:22 2022] POST /identity/v3/roles => generated 223 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:14:22.453535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b276cf4-a581-46e1-88a1-e2ebb1be1413 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:22.456566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fab5a-eacd-4964-b244-cc734a04f35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:22.458155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fab5a-eacd-4964-b244-cc734a04f35b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:22.458439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fab5a-eacd-4964-b244-cc734a04f35b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:22.458714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fab5a-eacd-4964-b244-cc734a04f35b None admin] PATH_INFO: `/v3/users/8a7b708ff6c140a3ba833eb00eff7926` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:22.459985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fab5a-eacd-4964-b244-cc734a04f35b None admin] RBAC: Authorizing `identity:get_user(user_id=8a7b708ff6c140a3ba833eb00eff7926)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:22.461530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fab5a-eacd-4964-b244-cc734a04f35b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:22.463924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 22/43] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:22 2022] GET /identity/v3/users/8a7b708ff6c140a3ba833eb00eff7926 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:22.469979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fab5a-eacd-4964-b244-cc734a04f35b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:22.542729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:22.568672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b276cf4-a581-46e1-88a1-e2ebb1be1413 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:22.587506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fab5a-eacd-4964-b244-cc734a04f35b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:22.605795 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6bec1-f623-48b5-872b-d9a09acd4a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:22.606424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6bec1-f623-48b5-872b-d9a09acd4a06 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:22.606704 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6bec1-f623-48b5-872b-d9a09acd4a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:22.606978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6bec1-f623-48b5-872b-d9a09acd4a06 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:22.607643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6bec1-f623-48b5-872b-d9a09acd4a06 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:22.609153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6bec1-f623-48b5-872b-d9a09acd4a06 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:22.624505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360713c-4d13-4447-9003-9bed30cf9655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:22.625212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360713c-4d13-4447-9003-9bed30cf9655 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:22.625467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360713c-4d13-4447-9003-9bed30cf9655 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:22.625735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360713c-4d13-4447-9003-9bed30cf9655 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:22.626881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360713c-4d13-4447-9003-9bed30cf9655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:22.628338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360713c-4d13-4447-9003-9bed30cf9655 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:22.630022 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 23/44] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:22.635939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360713c-4d13-4447-9003-9bed30cf9655 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:22.640121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 22/45] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:14:22 2022] POST /identity/v3/projects => generated 312 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:14:22.758030 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360713c-4d13-4447-9003-9bed30cf9655 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:22.784937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:22.794555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe55f9-71e6-43c5-9469-a801a0d59e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:22.795113 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe55f9-71e6-43c5-9469-a801a0d59e20 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:22.795428 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe55f9-71e6-43c5-9469-a801a0d59e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:22.795709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe55f9-71e6-43c5-9469-a801a0d59e20 None admin] PATH_INFO: `/v3/domains/default/users/8a7b708ff6c140a3ba833eb00eff7926/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:22.797789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe55f9-71e6-43c5-9469-a801a0d59e20 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8a7b708ff6c140a3ba833eb00eff7926, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:22.799268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe55f9-71e6-43c5-9469-a801a0d59e20 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:22.815786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 24/46] 10.222.0.60 () {58 vars in 1361 bytes} [Sat Sep 17 04:14:22 2022] PUT /identity/v3/domains/default/users/8a7b708ff6c140a3ba833eb00eff7926/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:14:24.784141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543b389-c412-4de8-9d5e-4a69ac8bd332 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:24.784685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543b389-c412-4de8-9d5e-4a69ac8bd332 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:24.784685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543b389-c412-4de8-9d5e-4a69ac8bd332 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:24.786223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 23/47] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:24.791512 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264971c-1b50-4fac-bd00-e4120cadfaea None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:24.791930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264971c-1b50-4fac-bd00-e4120cadfaea None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:24.792144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264971c-1b50-4fac-bd00-e4120cadfaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:24.812805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a264971c-1b50-4fac-bd00-e4120cadfaea None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:24.814188 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a264971c-1b50-4fac-bd00-e4120cadfaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:24.885521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 25/48] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:24.923931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124278d-b9aa-4ae0-8f10-b258deda36e5 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:24.924212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124278d-b9aa-4ae0-8f10-b258deda36e5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:24.924552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124278d-b9aa-4ae0-8f10-b258deda36e5 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:24.925981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 24/49] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:24.931586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efc35a-874f-4dd5-b075-f93f3dc09628 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:24.931871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efc35a-874f-4dd5-b075-f93f3dc09628 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:24.932144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efc35a-874f-4dd5-b075-f93f3dc09628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:24.950492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f5efc35a-874f-4dd5-b075-f93f3dc09628 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:24.951664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f5efc35a-874f-4dd5-b075-f93f3dc09628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:25.018361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 26/50] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:14:25.154982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124278d-b9aa-4ae0-8f10-b258deda36e5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:25.191343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:25.257118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124278d-b9aa-4ae0-8f10-b258deda36e5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:25.281683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efc35a-874f-4dd5-b075-f93f3dc09628 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:25.289351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6b5de-6d53-4557-bba8-54a6aea22c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:25.290029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6b5de-6d53-4557-bba8-54a6aea22c2b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:25.290309 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6b5de-6d53-4557-bba8-54a6aea22c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:25.290567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6b5de-6d53-4557-bba8-54a6aea22c2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:25.291736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6b5de-6d53-4557-bba8-54a6aea22c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:25.293230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6b5de-6d53-4557-bba8-54a6aea22c2b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:25.294806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 25/51] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:25.300398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6b5de-6d53-4557-bba8-54a6aea22c2b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:25.331216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:25.384814 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efc35a-874f-4dd5-b075-f93f3dc09628 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:25.399407 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6b5de-6d53-4557-bba8-54a6aea22c2b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:25.400902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:25.418564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788576e-a283-4787-b61f-edebd2bdfbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:25.419203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788576e-a283-4787-b61f-edebd2bdfbe4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:25.419478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788576e-a283-4787-b61f-edebd2bdfbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:25.419748 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788576e-a283-4787-b61f-edebd2bdfbe4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:25.429417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788576e-a283-4787-b61f-edebd2bdfbe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:25.431820 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61f225-6c19-4395-8c7b-0bb4a49e1a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:25.432760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61f225-6c19-4395-8c7b-0bb4a49e1a46 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:25.433045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61f225-6c19-4395-8c7b-0bb4a49e1a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:25.433345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61f225-6c19-4395-8c7b-0bb4a49e1a46 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:25.433993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61f225-6c19-4395-8c7b-0bb4a49e1a46 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-f788576e-a283-4787-b61f-edebd2bdfbe4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:25.434241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:25.435242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:25.436231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 27/52] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:14:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:25.437015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61f225-6c19-4395-8c7b-0bb4a49e1a46 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:25.443213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788576e-a283-4787-b61f-edebd2bdfbe4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:25.463523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 26/53] 10.222.0.60 () {60 vars in 1140 bytes} [Sat Sep 17 04:14:25 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:14:25.550391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788576e-a283-4787-b61f-edebd2bdfbe4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:25.560949 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:25.587240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36d09d-f0ab-4e7b-aae5-72b4a85c0a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:25.587972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36d09d-f0ab-4e7b-aae5-72b4a85c0a2b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:25.588247 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36d09d-f0ab-4e7b-aae5-72b4a85c0a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:25.588517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36d09d-f0ab-4e7b-aae5-72b4a85c0a2b None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:25.589251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36d09d-f0ab-4e7b-aae5-72b4a85c0a2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:25.589551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36d09d-f0ab-4e7b-aae5-72b4a85c0a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:25.590971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36d09d-f0ab-4e7b-aae5-72b4a85c0a2b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:25.600206 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 28/54] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:25.606279 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61f225-6c19-4395-8c7b-0bb4a49e1a46 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:25.711487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61f225-6c19-4395-8c7b-0bb4a49e1a46 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:25.744749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dba250-bd77-4b61-9439-e23cb36245a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:25.745421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dba250-bd77-4b61-9439-e23cb36245a6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:25.745702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dba250-bd77-4b61-9439-e23cb36245a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:25.745974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dba250-bd77-4b61-9439-e23cb36245a6 None admin] PATH_INFO: `/v3/users/8a7b708ff6c140a3ba833eb00eff7926` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:25.747104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dba250-bd77-4b61-9439-e23cb36245a6 None admin] RBAC: Authorizing `identity:get_user(user_id=8a7b708ff6c140a3ba833eb00eff7926)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:25.748563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dba250-bd77-4b61-9439-e23cb36245a6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:25.750373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 27/55] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:25 2022] GET /identity/v3/users/8a7b708ff6c140a3ba833eb00eff7926 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:25.755902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36d09d-f0ab-4e7b-aae5-72b4a85c0a2b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:25.856301 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36d09d-f0ab-4e7b-aae5-72b4a85c0a2b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:25.890281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767ad18-d1c0-446c-9641-10c0502167a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:25.890904 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767ad18-d1c0-446c-9641-10c0502167a2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:25.891180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767ad18-d1c0-446c-9641-10c0502167a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:25.891447 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767ad18-d1c0-446c-9641-10c0502167a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:25.892593 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767ad18-d1c0-446c-9641-10c0502167a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:25.894055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767ad18-d1c0-446c-9641-10c0502167a2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:25.895635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 29/56] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:25.901719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dba250-bd77-4b61-9439-e23cb36245a6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:25.928963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:26.004654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dba250-bd77-4b61-9439-e23cb36245a6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:26.039432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262f97a-0df3-412b-8e40-729a4c92e6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:26.040052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262f97a-0df3-412b-8e40-729a4c92e6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:26.040331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262f97a-0df3-412b-8e40-729a4c92e6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:26.040606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262f97a-0df3-412b-8e40-729a4c92e6a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:26.041452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262f97a-0df3-412b-8e40-729a4c92e6a7 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.domain.id=default, user.id=8a7b708ff6c140a3ba833eb00eff7926) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:26.041766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262f97a-0df3-412b-8e40-729a4c92e6a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:26.043199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262f97a-0df3-412b-8e40-729a4c92e6a7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:26.049128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 28/57] 10.222.0.60 () {56 vars in 1322 bytes} [Sat Sep 17 04:14:25 2022] GET /identity/v3/role_assignments?user.id=8a7b708ff6c140a3ba833eb00eff7926&scope.domain.id=default&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:27.608828 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76130a3f-799c-4cb9-b063-868fd9c7ba8f None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:27.609420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76130a3f-799c-4cb9-b063-868fd9c7ba8f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:27.609420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76130a3f-799c-4cb9-b063-868fd9c7ba8f None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:27.610790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 30/58] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:27.617003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0579de-147d-420a-b48b-d630059ddb57 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:27.617843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0579de-147d-420a-b48b-d630059ddb57 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:27.618240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0579de-147d-420a-b48b-d630059ddb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:27.644134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8d0579de-147d-420a-b48b-d630059ddb57 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:27.645914 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8d0579de-147d-420a-b48b-d630059ddb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:27.717939 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 29/59] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:27.990641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76130a3f-799c-4cb9-b063-868fd9c7ba8f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:28.142538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:28.208329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76130a3f-799c-4cb9-b063-868fd9c7ba8f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:28.244560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b33b3-ebdb-4eaf-a918-27871de787fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:28.245129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b33b3-ebdb-4eaf-a918-27871de787fc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:28.245422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b33b3-ebdb-4eaf-a918-27871de787fc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:28.245692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b33b3-ebdb-4eaf-a918-27871de787fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:28.246872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b33b3-ebdb-4eaf-a918-27871de787fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:28.248287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b33b3-ebdb-4eaf-a918-27871de787fc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:28.249910 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 31/60] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:28.255851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0579de-147d-420a-b48b-d630059ddb57 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:28.274802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:28.382209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0579de-147d-420a-b48b-d630059ddb57 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:28.419483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43452c0-e8f8-4c36-a972-2d0440914681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:28.420095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43452c0-e8f8-4c36-a972-2d0440914681 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:28.420377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43452c0-e8f8-4c36-a972-2d0440914681 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:28.420648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43452c0-e8f8-4c36-a972-2d0440914681 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:28.421326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43452c0-e8f8-4c36-a972-2d0440914681 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:28.422775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43452c0-e8f8-4c36-a972-2d0440914681 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:28.452476 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 30/61] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:14:28 2022] POST /identity/v3/users => generated 275 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:14:31.645716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45f1df-86d8-4d22-a6cd-a6e714a03864 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:31.646750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45f1df-86d8-4d22-a6cd-a6e714a03864 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:31.647221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45f1df-86d8-4d22-a6cd-a6e714a03864 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:31.648800 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 32/62] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:31.654345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34d2f6-5381-4cba-95a8-20b0366c3b19 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:31.655134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34d2f6-5381-4cba-95a8-20b0366c3b19 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:31.655599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34d2f6-5381-4cba-95a8-20b0366c3b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:31.681589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9f34d2f6-5381-4cba-95a8-20b0366c3b19 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:31.682798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9f34d2f6-5381-4cba-95a8-20b0366c3b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:31.776155 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 31/63] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:14:31.887623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e31454-dad8-405e-bf39-57d2bdb8fa67 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:31.888255 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e31454-dad8-405e-bf39-57d2bdb8fa67 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:31.888748 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e31454-dad8-405e-bf39-57d2bdb8fa67 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:31.890366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 33/64] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:31.895908 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06589221-881c-4f74-a7c1-625924b34b74 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:31.896138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06589221-881c-4f74-a7c1-625924b34b74 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:31.896421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06589221-881c-4f74-a7c1-625924b34b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:31.902955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:31.922134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-06589221-881c-4f74-a7c1-625924b34b74 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:31.923382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-06589221-881c-4f74-a7c1-625924b34b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:32.019522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 32/65] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:32.059858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640e227-c492-45cd-8818-a777a2bab36a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:32.060493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640e227-c492-45cd-8818-a777a2bab36a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:32.060993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640e227-c492-45cd-8818-a777a2bab36a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:32.062634 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 34/66] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:32.068874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9aecaf-08f6-469f-88d3-dca7c9505dbd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:32.069215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9aecaf-08f6-469f-88d3-dca7c9505dbd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:32.069454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9aecaf-08f6-469f-88d3-dca7c9505dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:32.094574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5e9aecaf-08f6-469f-88d3-dca7c9505dbd None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:32.095800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5e9aecaf-08f6-469f-88d3-dca7c9505dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:32.137256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640e227-c492-45cd-8818-a777a2bab36a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:32.193073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 33/67] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:32.194691 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef59b08-d56f-45b5-90d3-78d40206f43b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:32.195194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef59b08-d56f-45b5-90d3-78d40206f43b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:32.195663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef59b08-d56f-45b5-90d3-78d40206f43b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:32.197386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 34/68] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:32.208513 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ab762-9e22-4607-adeb-7766f14feab9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:32.209161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ab762-9e22-4607-adeb-7766f14feab9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:32.209743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ab762-9e22-4607-adeb-7766f14feab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:32.242986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8c1ab762-9e22-4607-adeb-7766f14feab9 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:32.244572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8c1ab762-9e22-4607-adeb-7766f14feab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:32.263783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640e227-c492-45cd-8818-a777a2bab36a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:32.275481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:32.333496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdab2c-d2f3-4228-8412-854444549cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:32.334196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdab2c-d2f3-4228-8412-854444549cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:32.334473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdab2c-d2f3-4228-8412-854444549cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:32.334741 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdab2c-d2f3-4228-8412-854444549cd2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:32.350088 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdab2c-d2f3-4228-8412-854444549cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:32.354279 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-97cdab2c-d2f3-4228-8412-854444549cd2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:32.355227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:32.356196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:32.356958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:32.356958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:32.356958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:32.356958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:32.356958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:32.356958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:32.356958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:32.356958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:32.356958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:32.356958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:32.356958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:32.356958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:32.359240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 35/69] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:14:32.360267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdab2c-d2f3-4228-8412-854444549cd2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:32.373197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 35/70] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:32.374344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ab762-9e22-4607-adeb-7766f14feab9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:32.502103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdab2c-d2f3-4228-8412-854444549cd2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:32.506818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ab762-9e22-4607-adeb-7766f14feab9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:32.538793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266b245-bab6-4866-962d-fa1784872dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:32.539741 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266b245-bab6-4866-962d-fa1784872dae None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:32.540341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266b245-bab6-4866-962d-fa1784872dae None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:32.540605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805d293-2930-45e9-9174-9d478b0cdcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:32.540950 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266b245-bab6-4866-962d-fa1784872dae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:32.541517 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805d293-2930-45e9-9174-9d478b0cdcc4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:32.541896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805d293-2930-45e9-9174-9d478b0cdcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:32.542259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805d293-2930-45e9-9174-9d478b0cdcc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:32.543034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805d293-2930-45e9-9174-9d478b0cdcc4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:32.543448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805d293-2930-45e9-9174-9d478b0cdcc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:32.544921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805d293-2930-45e9-9174-9d478b0cdcc4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:32.553698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 36/71] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:14:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:32.554763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805d293-2930-45e9-9174-9d478b0cdcc4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:32.555171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266b245-bab6-4866-962d-fa1784872dae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-1266b245-bab6-4866-962d-fa1784872dae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:32.561858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:32.562780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:32.563559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:32.563559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:32.563559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:32.563559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:32.563559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:32.563559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:32.563559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:32.563559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:32.563559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:32.563559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:32.563559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:32.563559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:32.566494 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 36/72] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:14:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:32.572729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266b245-bab6-4866-962d-fa1784872dae None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:32.604733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:32.614959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:32.659517 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805d293-2930-45e9-9174-9d478b0cdcc4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:32.684641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266b245-bab6-4866-962d-fa1784872dae None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:32.693926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d66605-b5ac-4220-b176-7d6b1d0d4cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:32.694573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d66605-b5ac-4220-b176-7d6b1d0d4cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:32.694863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d66605-b5ac-4220-b176-7d6b1d0d4cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:32.695128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d66605-b5ac-4220-b176-7d6b1d0d4cd5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:32.706796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d66605-b5ac-4220-b176-7d6b1d0d4cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-89d66605-b5ac-4220-b176-7d6b1d0d4cd5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:32.712700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:32.713616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:14:32.714363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 37/73] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:14:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:32.714981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d66605-b5ac-4220-b176-7d6b1d0d4cd5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:32.720014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20de02-dd51-4e0f-98bf-24247d9cb45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:32.720014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20de02-dd51-4e0f-98bf-24247d9cb45f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:32.720014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20de02-dd51-4e0f-98bf-24247d9cb45f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:32.720014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20de02-dd51-4e0f-98bf-24247d9cb45f None admin] PATH_INFO: `/v3/users/f361cc03b1bf40ce81b648b897538d31` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:32.735303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20de02-dd51-4e0f-98bf-24247d9cb45f None admin] RBAC: Authorizing `identity:get_user(user_id=f361cc03b1bf40ce81b648b897538d31)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:32.736757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20de02-dd51-4e0f-98bf-24247d9cb45f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:32.738440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 37/74] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:32 2022] GET /identity/v3/users/f361cc03b1bf40ce81b648b897538d31 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:32.739203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20de02-dd51-4e0f-98bf-24247d9cb45f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:32.813741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d66605-b5ac-4220-b176-7d6b1d0d4cd5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:32.821770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:32.827759 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:32.838725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20de02-dd51-4e0f-98bf-24247d9cb45f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:32.848375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77facadf-efcd-4592-b861-2c2cc4450d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:32.848991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77facadf-efcd-4592-b861-2c2cc4450d87 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:32.849294 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77facadf-efcd-4592-b861-2c2cc4450d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:32.849563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77facadf-efcd-4592-b861-2c2cc4450d87 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:32.850224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77facadf-efcd-4592-b861-2c2cc4450d87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:32.850525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77facadf-efcd-4592-b861-2c2cc4450d87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:32.851919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77facadf-efcd-4592-b861-2c2cc4450d87 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:32.861162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 38/75] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:32.862383 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159e088-1bc2-46fb-bb21-e2b4778a1f03 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:32.862667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159e088-1bc2-46fb-bb21-e2b4778a1f03 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:32.862936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159e088-1bc2-46fb-bb21-e2b4778a1f03 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:32.868912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 39/76] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:32.868912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159e088-1bc2-46fb-bb21-e2b4778a1f03 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:32.877219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a12af9-80c3-4462-8944-6f38783cf0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:32.877887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a12af9-80c3-4462-8944-6f38783cf0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:32.878167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a12af9-80c3-4462-8944-6f38783cf0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:32.878444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a12af9-80c3-4462-8944-6f38783cf0d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:32.888550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a12af9-80c3-4462-8944-6f38783cf0d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b6a12af9-80c3-4462-8944-6f38783cf0d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:32.893346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:32.894253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:32.895026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 38/77] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:32.895669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a12af9-80c3-4462-8944-6f38783cf0d2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:32.972900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159e088-1bc2-46fb-bb21-e2b4778a1f03 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:32.992852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a12af9-80c3-4462-8944-6f38783cf0d2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:33.006642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178a513-987d-4a4e-a36e-81548c46734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:33.007247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178a513-987d-4a4e-a36e-81548c46734c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:33.007529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178a513-987d-4a4e-a36e-81548c46734c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:33.007832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178a513-987d-4a4e-a36e-81548c46734c None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:33.008487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178a513-987d-4a4e-a36e-81548c46734c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:33.008773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178a513-987d-4a4e-a36e-81548c46734c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:33.010316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178a513-987d-4a4e-a36e-81548c46734c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:33.017935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 40/78] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:14:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:33.023932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36326a26-d249-4994-ba8d-b5a62424b164 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:33.024217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36326a26-d249-4994-ba8d-b5a62424b164 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:33.024493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36326a26-d249-4994-ba8d-b5a62424b164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:33.026539 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41d55f-0719-4f4c-8e91-6c48f573fa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:33.027156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41d55f-0719-4f4c-8e91-6c48f573fa39 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:33.027439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41d55f-0719-4f4c-8e91-6c48f573fa39 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:33.027713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41d55f-0719-4f4c-8e91-6c48f573fa39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:33.041908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:33.042422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-36326a26-d249-4994-ba8d-b5a62424b164 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:33.043710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-36326a26-d249-4994-ba8d-b5a62424b164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:33.073186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:33.096187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41d55f-0719-4f4c-8e91-6c48f573fa39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:33.097688 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41d55f-0719-4f4c-8e91-6c48f573fa39 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:33.108802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 41/79] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:33.109556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36326a26-d249-4994-ba8d-b5a62424b164 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:33.208992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36326a26-d249-4994-ba8d-b5a62424b164 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:33.216366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 39/80] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:33.217261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41d55f-0719-4f4c-8e91-6c48f573fa39 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:33.252149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde8c13-95e2-467c-a933-be6303331b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:33.253128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde8c13-95e2-467c-a933-be6303331b65 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:33.253622 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde8c13-95e2-467c-a933-be6303331b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:33.254089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde8c13-95e2-467c-a933-be6303331b65 None admin] PATH_INFO: `/v3/users/8a7b708ff6c140a3ba833eb00eff7926` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:33.255394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde8c13-95e2-467c-a933-be6303331b65 None admin] RBAC: Authorizing `identity:get_user(user_id=8a7b708ff6c140a3ba833eb00eff7926)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:33.257165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde8c13-95e2-467c-a933-be6303331b65 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:33.258956 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 42/81] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:33 2022] GET /identity/v3/users/8a7b708ff6c140a3ba833eb00eff7926 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:33.260055 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde8c13-95e2-467c-a933-be6303331b65 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:33.265566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:33.299874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:33.321196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41d55f-0719-4f4c-8e91-6c48f573fa39 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:33.354306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af173122-8dd8-4019-bd1a-6331e002c380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:33.354850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af173122-8dd8-4019-bd1a-6331e002c380 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:33.355123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af173122-8dd8-4019-bd1a-6331e002c380 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:33.355391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af173122-8dd8-4019-bd1a-6331e002c380 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:33.356059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af173122-8dd8-4019-bd1a-6331e002c380 None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:33.356357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af173122-8dd8-4019-bd1a-6331e002c380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:33.357764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af173122-8dd8-4019-bd1a-6331e002c380 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:33.365717 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 40/82] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:14:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:33.366482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af173122-8dd8-4019-bd1a-6331e002c380 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:33.372778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde8c13-95e2-467c-a933-be6303331b65 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:33.406156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0434fb-96c5-432c-919f-7bb4fc6847c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:33.406786 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0434fb-96c5-432c-919f-7bb4fc6847c8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:33.407063 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0434fb-96c5-432c-919f-7bb4fc6847c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:33.407307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0434fb-96c5-432c-919f-7bb4fc6847c8 None admin] PATH_INFO: `/v3/users/f361cc03b1bf40ce81b648b897538d31` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:33.408432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0434fb-96c5-432c-919f-7bb4fc6847c8 None admin] RBAC: Authorizing `identity:get_user(user_id=f361cc03b1bf40ce81b648b897538d31)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:33.409906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0434fb-96c5-432c-919f-7bb4fc6847c8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:33.411511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 43/83] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:33 2022] GET /identity/v3/users/f361cc03b1bf40ce81b648b897538d31 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:33.416920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0434fb-96c5-432c-919f-7bb4fc6847c8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:33.465979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af173122-8dd8-4019-bd1a-6331e002c380 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:33.472633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:33.499554 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93b9b7-cf2c-4db4-8fa1-380a8453f3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:33.500190 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93b9b7-cf2c-4db4-8fa1-380a8453f3dd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:33.500465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93b9b7-cf2c-4db4-8fa1-380a8453f3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:33.500733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93b9b7-cf2c-4db4-8fa1-380a8453f3dd None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:33.501889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93b9b7-cf2c-4db4-8fa1-380a8453f3dd None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:33.503322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93b9b7-cf2c-4db4-8fa1-380a8453f3dd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:33.504944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 41/84] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:33 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:33.510263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93b9b7-cf2c-4db4-8fa1-380a8453f3dd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:33.514420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0434fb-96c5-432c-919f-7bb4fc6847c8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:33.516707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:33.548340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66a670-e35c-4bea-a01b-e8580794172a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:33.548956 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66a670-e35c-4bea-a01b-e8580794172a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:33.549263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66a670-e35c-4bea-a01b-e8580794172a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:33.549531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66a670-e35c-4bea-a01b-e8580794172a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:33.603663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93b9b7-cf2c-4db4-8fa1-380a8453f3dd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:33.616087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66a670-e35c-4bea-a01b-e8580794172a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:33.618061 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66a670-e35c-4bea-a01b-e8580794172a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:33.636120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff34fe-3b23-42db-93ac-66d97e097b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:33.636930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff34fe-3b23-42db-93ac-66d97e097b25 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:33.637244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff34fe-3b23-42db-93ac-66d97e097b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:33.637517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff34fe-3b23-42db-93ac-66d97e097b25 None admin] PATH_INFO: `/v3/users/f361cc03b1bf40ce81b648b897538d31` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:33.638651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff34fe-3b23-42db-93ac-66d97e097b25 None admin] RBAC: Authorizing `identity:get_user(user_id=f361cc03b1bf40ce81b648b897538d31)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:33.640077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff34fe-3b23-42db-93ac-66d97e097b25 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:33.641723 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 42/85] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:33 2022] GET /identity/v3/users/f361cc03b1bf40ce81b648b897538d31 => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:33.646231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff34fe-3b23-42db-93ac-66d97e097b25 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:33.687626 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:33.716461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 44/86] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:33.717352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66a670-e35c-4bea-a01b-e8580794172a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:33.720321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:33.746133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff34fe-3b23-42db-93ac-66d97e097b25 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:33.780192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1015ac5-7c4c-4233-b0b9-d7be4150b8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:33.780822 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1015ac5-7c4c-4233-b0b9-d7be4150b8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:33.781109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1015ac5-7c4c-4233-b0b9-d7be4150b8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:33.781417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1015ac5-7c4c-4233-b0b9-d7be4150b8f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:33.782553 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1015ac5-7c4c-4233-b0b9-d7be4150b8f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:33.783988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1015ac5-7c4c-4233-b0b9-d7be4150b8f2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:33.785580 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 43/87] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:33.786345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1015ac5-7c4c-4233-b0b9-d7be4150b8f2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:33.816284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66a670-e35c-4bea-a01b-e8580794172a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:33.849324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27c4fe-b93d-4e0f-97ee-dec9f13cb334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:33.849933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27c4fe-b93d-4e0f-97ee-dec9f13cb334 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:33.850208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27c4fe-b93d-4e0f-97ee-dec9f13cb334 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:33.850478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27c4fe-b93d-4e0f-97ee-dec9f13cb334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:33.880965 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1015ac5-7c4c-4233-b0b9-d7be4150b8f2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:33.890842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:33.915996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab9af5-ab23-4c2a-9339-760de5ddbe7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:33.916607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab9af5-ab23-4c2a-9339-760de5ddbe7e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:33.916786 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27c4fe-b93d-4e0f-97ee-dec9f13cb334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:33.916887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab9af5-ab23-4c2a-9339-760de5ddbe7e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:33.917187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab9af5-ab23-4c2a-9339-760de5ddbe7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:33.917911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab9af5-ab23-4c2a-9339-760de5ddbe7e None admin] RBAC: Adding query filter params (role.id=231a681c35614ea3aaf3452a8c167c5c, scope.project.id=d7203986da504ab1ba9f76d7f64f7dce, user.id=f361cc03b1bf40ce81b648b897538d31) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:33.918219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab9af5-ab23-4c2a-9339-760de5ddbe7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:33.918307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27c4fe-b93d-4e0f-97ee-dec9f13cb334 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:33.919627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab9af5-ab23-4c2a-9339-760de5ddbe7e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:33.921380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:33.927203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 44/88] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:33 2022] GET /identity/v3/role_assignments?user.id=f361cc03b1bf40ce81b648b897538d31&scope.project.id=d7203986da504ab1ba9f76d7f64f7dce&role.id=231a681c35614ea3aaf3452a8c167c5c => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:33.928083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab9af5-ab23-4c2a-9339-760de5ddbe7e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:34.032830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 45/89] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:34.038633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27c4fe-b93d-4e0f-97ee-dec9f13cb334 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:34.038953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab9af5-ab23-4c2a-9339-760de5ddbe7e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:34.076911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a6278-b490-44a6-87ae-799907018e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:34.077579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a6278-b490-44a6-87ae-799907018e1f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:34.077865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a6278-b490-44a6-87ae-799907018e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:34.078140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a6278-b490-44a6-87ae-799907018e1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:34.129803 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:34.151458 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27c4fe-b93d-4e0f-97ee-dec9f13cb334 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:34.154960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:34.156903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a6278-b490-44a6-87ae-799907018e1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:34.158519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a6278-b490-44a6-87ae-799907018e1f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:34.187250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6ce41-2b45-4a72-9edc-d66009863773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:34.187896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6ce41-2b45-4a72-9edc-d66009863773 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:34.188207 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6ce41-2b45-4a72-9edc-d66009863773 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:34.188443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6ce41-2b45-4a72-9edc-d66009863773 None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:34.189662 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6ce41-2b45-4a72-9edc-d66009863773 None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:34.191222 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6ce41-2b45-4a72-9edc-d66009863773 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:34.197648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 46/90] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:34 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:34.197648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6ce41-2b45-4a72-9edc-d66009863773 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:34.265186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 45/91] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:34.266188 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a6278-b490-44a6-87ae-799907018e1f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:34.301378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6ce41-2b45-4a72-9edc-d66009863773 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:34.335442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f0849-3979-4319-b282-d7b7b9ba5e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:34.336358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f0849-3979-4319-b282-d7b7b9ba5e89 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:34.336808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f0849-3979-4319-b282-d7b7b9ba5e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:34.337378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f0849-3979-4319-b282-d7b7b9ba5e89 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:34.338405 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f0849-3979-4319-b282-d7b7b9ba5e89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:34.339960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f0849-3979-4319-b282-d7b7b9ba5e89 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:34.342810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:34.369215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 47/92] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:14:34 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:14:34.369215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f0849-3979-4319-b282-d7b7b9ba5e89 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:34.381625 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a6278-b490-44a6-87ae-799907018e1f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:34.418899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:34.425896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e98535-8d31-4911-8e8a-8904e5eeb820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:34.427208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e98535-8d31-4911-8e8a-8904e5eeb820 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:34.427749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e98535-8d31-4911-8e8a-8904e5eeb820 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:34.428171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e98535-8d31-4911-8e8a-8904e5eeb820 None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:34.429877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e98535-8d31-4911-8e8a-8904e5eeb820 None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:34.431781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e98535-8d31-4911-8e8a-8904e5eeb820 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:34.434356 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 46/93] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:34 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:34.435573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e98535-8d31-4911-8e8a-8904e5eeb820 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:34.527668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f0849-3979-4319-b282-d7b7b9ba5e89 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:34.556171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e98535-8d31-4911-8e8a-8904e5eeb820 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:34.578943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89906364-9cc0-4cd0-bbc4-066c53263eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:34.579579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89906364-9cc0-4cd0-bbc4-066c53263eae None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:34.579850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89906364-9cc0-4cd0-bbc4-066c53263eae None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:34.580120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89906364-9cc0-4cd0-bbc4-066c53263eae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:34.580869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89906364-9cc0-4cd0-bbc4-066c53263eae None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=d7203986da504ab1ba9f76d7f64f7dce, user.id=8a7b708ff6c140a3ba833eb00eff7926) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:34.581224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89906364-9cc0-4cd0-bbc4-066c53263eae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:34.582762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89906364-9cc0-4cd0-bbc4-066c53263eae None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:34.588640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 48/94] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:34 2022] GET /identity/v3/role_assignments?user.id=8a7b708ff6c140a3ba833eb00eff7926&scope.project.id=d7203986da504ab1ba9f76d7f64f7dce&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:34.589707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89906364-9cc0-4cd0-bbc4-066c53263eae None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:34.607372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14391e4c-4925-4c64-bf39-fe659613a91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:34.608047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14391e4c-4925-4c64-bf39-fe659613a91b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:34.608343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14391e4c-4925-4c64-bf39-fe659613a91b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:34.608607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14391e4c-4925-4c64-bf39-fe659613a91b None admin] PATH_INFO: `/v3/projects/498b5a50ecb147c8b5c7f428dddc9c4b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:34.614035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14391e4c-4925-4c64-bf39-fe659613a91b None admin] RBAC: Authorizing `identity:get_project(project_id=498b5a50ecb147c8b5c7f428dddc9c4b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:34.615541 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14391e4c-4925-4c64-bf39-fe659613a91b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:34.617704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 47/95] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:34 2022] GET /identity/v3/projects/498b5a50ecb147c8b5c7f428dddc9c4b => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:34.623686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14391e4c-4925-4c64-bf39-fe659613a91b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:34.645706 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:34.713990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:34.726699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89906364-9cc0-4cd0-bbc4-066c53263eae None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:34.774001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14391e4c-4925-4c64-bf39-fe659613a91b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:34.776389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe0542-162b-472f-b69d-9cec00a9fc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:34.778187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe0542-162b-472f-b69d-9cec00a9fc80 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:34.778586 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe0542-162b-472f-b69d-9cec00a9fc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:34.778987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe0542-162b-472f-b69d-9cec00a9fc80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:34.779853 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe0542-162b-472f-b69d-9cec00a9fc80 None admin] RBAC: Adding query filter params (role.id=84212031716c4cf8a5083824fe47dbb7, scope.project.id=d7203986da504ab1ba9f76d7f64f7dce, user.id=f361cc03b1bf40ce81b648b897538d31) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:34.780265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe0542-162b-472f-b69d-9cec00a9fc80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:34.781948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe0542-162b-472f-b69d-9cec00a9fc80 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:34.791551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 49/96] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:34 2022] GET /identity/v3/role_assignments?user.id=f361cc03b1bf40ce81b648b897538d31&scope.project.id=d7203986da504ab1ba9f76d7f64f7dce&role.id=84212031716c4cf8a5083824fe47dbb7 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:34.842200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0323d-f8de-4835-9b59-c7f06f0e2383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:34.842840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0323d-f8de-4835-9b59-c7f06f0e2383 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:34.843115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0323d-f8de-4835-9b59-c7f06f0e2383 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:34.843384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0323d-f8de-4835-9b59-c7f06f0e2383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:34.844198 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0323d-f8de-4835-9b59-c7f06f0e2383 None admin] RBAC: Adding query filter params (role.id=231a681c35614ea3aaf3452a8c167c5c, scope.project.id=498b5a50ecb147c8b5c7f428dddc9c4b, user.id=f361cc03b1bf40ce81b648b897538d31) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:34.844511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0323d-f8de-4835-9b59-c7f06f0e2383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:34.846047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0323d-f8de-4835-9b59-c7f06f0e2383 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:34.852941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 48/97] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:34 2022] GET /identity/v3/role_assignments?user.id=f361cc03b1bf40ce81b648b897538d31&scope.project.id=498b5a50ecb147c8b5c7f428dddc9c4b&role.id=231a681c35614ea3aaf3452a8c167c5c => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:36.230717 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d89aa-44fb-469b-a2ec-9d5e3da8c48d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:36.231634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d89aa-44fb-469b-a2ec-9d5e3da8c48d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:36.232166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d89aa-44fb-469b-a2ec-9d5e3da8c48d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:36.234066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 50/98] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:36.239333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab14de0-d947-4ee5-8afb-fbe2ecaa6109 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:36.239731 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab14de0-d947-4ee5-8afb-fbe2ecaa6109 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:36.239942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab14de0-d947-4ee5-8afb-fbe2ecaa6109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:36.261976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1ab14de0-d947-4ee5-8afb-fbe2ecaa6109 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:36.263233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1ab14de0-d947-4ee5-8afb-fbe2ecaa6109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:36.334524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 49/99] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:36.866730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d89aa-44fb-469b-a2ec-9d5e3da8c48d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:36.997862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d89aa-44fb-469b-a2ec-9d5e3da8c48d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:37.038780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63815b2-cc6e-4786-b681-d3db53020939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:37.039423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63815b2-cc6e-4786-b681-d3db53020939 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:37.039718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63815b2-cc6e-4786-b681-d3db53020939 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:37.039976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63815b2-cc6e-4786-b681-d3db53020939 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:37.059736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63815b2-cc6e-4786-b681-d3db53020939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b63815b2-cc6e-4786-b681-d3db53020939 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:37.064998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:37.065875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:14:37.066644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 51/100] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:37.073311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab14de0-d947-4ee5-8afb-fbe2ecaa6109 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:37.083139 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:37.248339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab14de0-d947-4ee5-8afb-fbe2ecaa6109 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:37.292424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0f87b-435e-4c24-b1d8-9615c715ffb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:37.293048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0f87b-435e-4c24-b1d8-9615c715ffb2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:37.293391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0f87b-435e-4c24-b1d8-9615c715ffb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:37.293659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0f87b-435e-4c24-b1d8-9615c715ffb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:37.294332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0f87b-435e-4c24-b1d8-9615c715ffb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:37.294631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0f87b-435e-4c24-b1d8-9615c715ffb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:37.296050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0f87b-435e-4c24-b1d8-9615c715ffb2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:37.307392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 50/101] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:14:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:37.313301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63815b2-cc6e-4786-b681-d3db53020939 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:37.348783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:37.460643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63815b2-cc6e-4786-b681-d3db53020939 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:37.508239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78558b6-2dc4-4000-b882-3d03f5240cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:37.508808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78558b6-2dc4-4000-b882-3d03f5240cbc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:37.509127 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78558b6-2dc4-4000-b882-3d03f5240cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:37.509407 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78558b6-2dc4-4000-b882-3d03f5240cbc None admin] PATH_INFO: `/v3/users/f361cc03b1bf40ce81b648b897538d31` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:37.510644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78558b6-2dc4-4000-b882-3d03f5240cbc None admin] RBAC: Authorizing `identity:get_user(user_id=f361cc03b1bf40ce81b648b897538d31)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:37.512103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78558b6-2dc4-4000-b882-3d03f5240cbc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:37.513743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 52/102] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:37 2022] GET /identity/v3/users/f361cc03b1bf40ce81b648b897538d31 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:37.519540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0f87b-435e-4c24-b1d8-9615c715ffb2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:37.654291 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:37.704881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0f87b-435e-4c24-b1d8-9615c715ffb2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:37.750794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84901db4-d5a3-4b83-9be2-93859cefb43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:37.751438 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84901db4-d5a3-4b83-9be2-93859cefb43f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:37.751712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84901db4-d5a3-4b83-9be2-93859cefb43f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:37.751980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84901db4-d5a3-4b83-9be2-93859cefb43f None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:37.756804 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84901db4-d5a3-4b83-9be2-93859cefb43f None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:37.758355 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84901db4-d5a3-4b83-9be2-93859cefb43f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:37.759989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 51/103] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:37 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:37.766022 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78558b6-2dc4-4000-b882-3d03f5240cbc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:37.906986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:37.925308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78558b6-2dc4-4000-b882-3d03f5240cbc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:37.931120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f48076-6d9c-4c36-966a-c88d61782d71 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:37.931514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f48076-6d9c-4c36-966a-c88d61782d71 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:37.931787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f48076-6d9c-4c36-966a-c88d61782d71 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:37.933212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 52/104] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:37.943701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06420bc-7662-4d9d-ae6e-be4b7449aa6b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:37.944004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06420bc-7662-4d9d-ae6e-be4b7449aa6b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:37.944261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06420bc-7662-4d9d-ae6e-be4b7449aa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:37.976963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d06420bc-7662-4d9d-ae6e-be4b7449aa6b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:37.981539 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d06420bc-7662-4d9d-ae6e-be4b7449aa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:37.998713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91e4fb-0f68-4e71-9499-1ea356e18387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:37.999393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91e4fb-0f68-4e71-9499-1ea356e18387 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:37.999667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91e4fb-0f68-4e71-9499-1ea356e18387 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:37.999935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91e4fb-0f68-4e71-9499-1ea356e18387 None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce/users/f361cc03b1bf40ce81b648b897538d31/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:38.003227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91e4fb-0f68-4e71-9499-1ea356e18387 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7203986da504ab1ba9f76d7f64f7dce, user_id=f361cc03b1bf40ce81b648b897538d31, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:38.004737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91e4fb-0f68-4e71-9499-1ea356e18387 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:38.031696 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 53/105] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:14:37 2022] PUT /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce/users/f361cc03b1bf40ce81b648b897538d31/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:14:38.033039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d330fcc-1d8d-4bac-ae14-6a6fb322aaf7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:38.033341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d330fcc-1d8d-4bac-ae14-6a6fb322aaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:38.033619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d330fcc-1d8d-4bac-ae14-6a6fb322aaf7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:38.035062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 54/106] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:38.041408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff151656-4a45-406a-aa43-bfc3e83a0359 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:38.041706 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff151656-4a45-406a-aa43-bfc3e83a0359 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:38.041954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff151656-4a45-406a-aa43-bfc3e83a0359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:38.082269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ff151656-4a45-406a-aa43-bfc3e83a0359 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:38.083511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ff151656-4a45-406a-aa43-bfc3e83a0359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:38.091189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 53/107] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:14:38.117147 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa74799-3851-421f-acd6-dec4cacc5354 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:38.117877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa74799-3851-421f-acd6-dec4cacc5354 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:38.118155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa74799-3851-421f-acd6-dec4cacc5354 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:38.119751 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 54/108] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:14:38.126490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64501476-74a3-4df2-8cce-f1610b97e07b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:38.126822 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64501476-74a3-4df2-8cce-f1610b97e07b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:38.127063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64501476-74a3-4df2-8cce-f1610b97e07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:38.166435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-64501476-74a3-4df2-8cce-f1610b97e07b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:38.167714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-64501476-74a3-4df2-8cce-f1610b97e07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:38.200143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 55/109] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:38.266892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:38.278786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 55/110] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:38.324194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485811d-800d-4507-978b-e49de1524535 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:38.324813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485811d-800d-4507-978b-e49de1524535 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:38.324813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485811d-800d-4507-978b-e49de1524535 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:38.326047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 56/111] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:38.331489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5d37f-b80e-465a-9b21-1d776b511490 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:38.331759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5d37f-b80e-465a-9b21-1d776b511490 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:38.332040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5d37f-b80e-465a-9b21-1d776b511490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:38.359860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-10c5d37f-b80e-465a-9b21-1d776b511490 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:38.361047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-10c5d37f-b80e-465a-9b21-1d776b511490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:38.375437 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453f7a0-4861-4545-8514-2c7bad867986 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:38.375739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453f7a0-4861-4545-8514-2c7bad867986 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:38.375982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453f7a0-4861-4545-8514-2c7bad867986 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:38.377545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 57/112] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:14:38.390164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5ff98-fc33-4b9e-be66-79c50f467971 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:38.390446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5ff98-fc33-4b9e-be66-79c50f467971 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:38.390722 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5ff98-fc33-4b9e-be66-79c50f467971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:38.430081 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2ae5ff98-fc33-4b9e-be66-79c50f467971 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:38.431328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2ae5ff98-fc33-4b9e-be66-79c50f467971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:38.443725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 56/113] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:38.444599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5d37f-b80e-465a-9b21-1d776b511490 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:38.594713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 58/114] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:14:38.595327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5ff98-fc33-4b9e-be66-79c50f467971 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:38.608621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5d37f-b80e-465a-9b21-1d776b511490 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:38.610190 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:38.656648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:38.689484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a8643-6502-46f3-8cbd-fc6085c79b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:38.690474 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a8643-6502-46f3-8cbd-fc6085c79b1a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:38.690952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a8643-6502-46f3-8cbd-fc6085c79b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:38.691414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a8643-6502-46f3-8cbd-fc6085c79b1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:38.709886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a8643-6502-46f3-8cbd-fc6085c79b1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c87a8643-6502-46f3-8cbd-fc6085c79b1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:38.718943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:38.719785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:38.720549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:38.720549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:38.720549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:38.720549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:38.720549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:38.720549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:38.720549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:38.720549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:38.720549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:38.720549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:38.720549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:38.720549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:38.725562 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 57/115] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:14:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:38.727386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a8643-6502-46f3-8cbd-fc6085c79b1a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:38.793355 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5ff98-fc33-4b9e-be66-79c50f467971 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:38.829740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a869b38-ea54-4404-98c3-05bdbe765eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:38.830752 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a869b38-ea54-4404-98c3-05bdbe765eed None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:38.831221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a869b38-ea54-4404-98c3-05bdbe765eed None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:38.831677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a869b38-ea54-4404-98c3-05bdbe765eed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:38.843413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a869b38-ea54-4404-98c3-05bdbe765eed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4a869b38-ea54-4404-98c3-05bdbe765eed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:38.849108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:38.850314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:38.851459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:38.851459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:38.851459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:38.851459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:38.851459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:38.851459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:38.851459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:38.851459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:38.851459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:38.851459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:38.851459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:14:38.851459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:14:38.859627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a8643-6502-46f3-8cbd-fc6085c79b1a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:38.859929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 59/116] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:38.861117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a869b38-ea54-4404-98c3-05bdbe765eed None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:38.902026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b71c65-f764-4525-9076-f125d9f3efc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:38.902664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b71c65-f764-4525-9076-f125d9f3efc2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:38.902945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b71c65-f764-4525-9076-f125d9f3efc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:38.903215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b71c65-f764-4525-9076-f125d9f3efc2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:38.916170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b71c65-f764-4525-9076-f125d9f3efc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-35b71c65-f764-4525-9076-f125d9f3efc2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:38.922506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:38.923335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:38.924109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 58/117] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:14:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:38.924726 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b71c65-f764-4525-9076-f125d9f3efc2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:38.936593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:38.972564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:38.978815 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a869b38-ea54-4404-98c3-05bdbe765eed None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:39.014530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d65e0-e1fe-4bee-97a2-7a2e25961f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:39.015250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d65e0-e1fe-4bee-97a2-7a2e25961f38 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:39.015522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d65e0-e1fe-4bee-97a2-7a2e25961f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:39.015784 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d65e0-e1fe-4bee-97a2-7a2e25961f38 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:39.016446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d65e0-e1fe-4bee-97a2-7a2e25961f38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:39.016742 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d65e0-e1fe-4bee-97a2-7a2e25961f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:39.018285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d65e0-e1fe-4bee-97a2-7a2e25961f38 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:39.027229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 60/118] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:39.027818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d65e0-e1fe-4bee-97a2-7a2e25961f38 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:39.032126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b71c65-f764-4525-9076-f125d9f3efc2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:39.068187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e9031-fc0c-47ad-a540-a93e1e71e88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:39.068853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e9031-fc0c-47ad-a540-a93e1e71e88b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:39.069127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e9031-fc0c-47ad-a540-a93e1e71e88b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:39.069394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e9031-fc0c-47ad-a540-a93e1e71e88b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:39.070608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e9031-fc0c-47ad-a540-a93e1e71e88b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:39.072087 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e9031-fc0c-47ad-a540-a93e1e71e88b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:39.074054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 59/119] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:39.074913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e9031-fc0c-47ad-a540-a93e1e71e88b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:39.135072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d65e0-e1fe-4bee-97a2-7a2e25961f38 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:39.161395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:39.171202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73617509-d482-4332-bcf2-70e81a17f513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:39.171826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73617509-d482-4332-bcf2-70e81a17f513 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:39.172103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73617509-d482-4332-bcf2-70e81a17f513 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:39.172369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73617509-d482-4332-bcf2-70e81a17f513 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:39.173037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73617509-d482-4332-bcf2-70e81a17f513 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:39.173364 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73617509-d482-4332-bcf2-70e81a17f513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:39.174755 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73617509-d482-4332-bcf2-70e81a17f513 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:39.183007 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e9031-fc0c-47ad-a540-a93e1e71e88b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:39.183482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 61/120] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:14:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:39.186798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73617509-d482-4332-bcf2-70e81a17f513 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:39.197116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:39.219892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d0334-f3a6-4b02-b9b5-84ebc905ab17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:39.220528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d0334-f3a6-4b02-b9b5-84ebc905ab17 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:39.220805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d0334-f3a6-4b02-b9b5-84ebc905ab17 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:39.221068 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d0334-f3a6-4b02-b9b5-84ebc905ab17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:39.231349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d0334-f3a6-4b02-b9b5-84ebc905ab17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d89d0334-f3a6-4b02-b9b5-84ebc905ab17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:39.236175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:39.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 60/121] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:39.242058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d0334-f3a6-4b02-b9b5-84ebc905ab17 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:39.295079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73617509-d482-4332-bcf2-70e81a17f513 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:39.334254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5e918-759c-4c7a-a686-171b874b0a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:39.334888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5e918-759c-4c7a-a686-171b874b0a9b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:39.335133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5e918-759c-4c7a-a686-171b874b0a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:39.335407 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5e918-759c-4c7a-a686-171b874b0a9b None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:39.336073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5e918-759c-4c7a-a686-171b874b0a9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:39.336371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5e918-759c-4c7a-a686-171b874b0a9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:39.337876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5e918-759c-4c7a-a686-171b874b0a9b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:39.343912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d0334-f3a6-4b02-b9b5-84ebc905ab17 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:39.346703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 62/122] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:14:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:39.347150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5e918-759c-4c7a-a686-171b874b0a9b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:39.385551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d38d22-ffae-4d69-b4d1-cb02261c53b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:39.386111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d38d22-ffae-4d69-b4d1-cb02261c53b7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:39.386387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d38d22-ffae-4d69-b4d1-cb02261c53b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:39.386654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d38d22-ffae-4d69-b4d1-cb02261c53b7 None admin] PATH_INFO: `/v3/users/8a7b708ff6c140a3ba833eb00eff7926` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:39.387882 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d38d22-ffae-4d69-b4d1-cb02261c53b7 None admin] RBAC: Authorizing `identity:get_user(user_id=8a7b708ff6c140a3ba833eb00eff7926)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:39.389360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d38d22-ffae-4d69-b4d1-cb02261c53b7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:39.391389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 61/123] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:39 2022] GET /identity/v3/users/8a7b708ff6c140a3ba833eb00eff7926 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:39.392156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d38d22-ffae-4d69-b4d1-cb02261c53b7 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:39.402622 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:39.435855 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:39.455718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5e918-759c-4c7a-a686-171b874b0a9b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:39.489769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176a35b-6d6c-4ada-9809-6a374252d33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:39.490392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176a35b-6d6c-4ada-9809-6a374252d33f None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:39.490665 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176a35b-6d6c-4ada-9809-6a374252d33f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:39.490945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176a35b-6d6c-4ada-9809-6a374252d33f None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:39.491605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176a35b-6d6c-4ada-9809-6a374252d33f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:39.493439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176a35b-6d6c-4ada-9809-6a374252d33f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:39.500077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d38d22-ffae-4d69-b4d1-cb02261c53b7 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:39.522669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 63/124] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:14:39 2022] POST /identity/v3/projects => generated 302 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:14:39.523516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176a35b-6d6c-4ada-9809-6a374252d33f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:39.553725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8964177-eca3-4e24-a018-58dff7da9f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:39.554651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8964177-eca3-4e24-a018-58dff7da9f7a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:39.555300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8964177-eca3-4e24-a018-58dff7da9f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:39.555650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8964177-eca3-4e24-a018-58dff7da9f7a None admin] PATH_INFO: `/v3/users/a0475613e7004333a8bd6e4342c3069e` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:39.577311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8964177-eca3-4e24-a018-58dff7da9f7a None admin] RBAC: Authorizing `identity:get_user(user_id=a0475613e7004333a8bd6e4342c3069e)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:39.579568 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8964177-eca3-4e24-a018-58dff7da9f7a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:39.582046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 62/125] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:39 2022] GET /identity/v3/users/a0475613e7004333a8bd6e4342c3069e => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:39.583018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8964177-eca3-4e24-a018-58dff7da9f7a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:39.664201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176a35b-6d6c-4ada-9809-6a374252d33f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:39.664560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:39.705926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd2142-a27d-4afe-8017-40738bdc3a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:39.707381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd2142-a27d-4afe-8017-40738bdc3a6a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:39.707664 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd2142-a27d-4afe-8017-40738bdc3a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:39.707996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd2142-a27d-4afe-8017-40738bdc3a6a None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:39.708621 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd2142-a27d-4afe-8017-40738bdc3a6a None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:39.708920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd2142-a27d-4afe-8017-40738bdc3a6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:39.709443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:39.712239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd2142-a27d-4afe-8017-40738bdc3a6a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:39.713471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8964177-eca3-4e24-a018-58dff7da9f7a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:39.720648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 64/126] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:14:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:39.725525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd2142-a27d-4afe-8017-40738bdc3a6a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:39.763067 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c2811-6f28-4c93-8f9d-4fa10f957cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:39.763711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c2811-6f28-4c93-8f9d-4fa10f957cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:39.763991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c2811-6f28-4c93-8f9d-4fa10f957cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:39.764259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c2811-6f28-4c93-8f9d-4fa10f957cc7 None admin] PATH_INFO: `/v3/users/f361cc03b1bf40ce81b648b897538d31` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:39.766445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c2811-6f28-4c93-8f9d-4fa10f957cc7 None admin] RBAC: Authorizing `identity:get_user(user_id=f361cc03b1bf40ce81b648b897538d31)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:39.767970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c2811-6f28-4c93-8f9d-4fa10f957cc7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:39.769608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 63/127] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:39 2022] GET /identity/v3/users/f361cc03b1bf40ce81b648b897538d31 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:39.770375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c2811-6f28-4c93-8f9d-4fa10f957cc7 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:39.856013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd2142-a27d-4afe-8017-40738bdc3a6a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:39.880193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c2811-6f28-4c93-8f9d-4fa10f957cc7 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:39.901960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2625e-b735-46c4-8dcc-b54e8fd28576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:39.902884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2625e-b735-46c4-8dcc-b54e8fd28576 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:39.903167 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2625e-b735-46c4-8dcc-b54e8fd28576 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:39.903445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2625e-b735-46c4-8dcc-b54e8fd28576 None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:39.905296 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2625e-b735-46c4-8dcc-b54e8fd28576 None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:39.907691 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2625e-b735-46c4-8dcc-b54e8fd28576 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:39.910057 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 65/128] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:39 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:39.913648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2625e-b735-46c4-8dcc-b54e8fd28576 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:39.921274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cf6b8-c5bb-4a6d-80d1-cd21f3b5eeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:39.921830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cf6b8-c5bb-4a6d-80d1-cd21f3b5eeaf None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:39.922103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cf6b8-c5bb-4a6d-80d1-cd21f3b5eeaf None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:39.922371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cf6b8-c5bb-4a6d-80d1-cd21f3b5eeaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:39.953653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:39.960093 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:39.996103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730cf6b8-c5bb-4a6d-80d1-cd21f3b5eeaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:39.997764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730cf6b8-c5bb-4a6d-80d1-cd21f3b5eeaf None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:40.040519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2625e-b735-46c4-8dcc-b54e8fd28576 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:40.078007 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c1569-445d-47e5-8914-668c75aad1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:40.078796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c1569-445d-47e5-8914-668c75aad1ad None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:40.079329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c1569-445d-47e5-8914-668c75aad1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:40.079710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c1569-445d-47e5-8914-668c75aad1ad None admin] PATH_INFO: `/v3/users/f361cc03b1bf40ce81b648b897538d31` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:40.081762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c1569-445d-47e5-8914-668c75aad1ad None admin] RBAC: Authorizing `identity:get_user(user_id=f361cc03b1bf40ce81b648b897538d31)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:40.083489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c1569-445d-47e5-8914-668c75aad1ad None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:40.086185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 66/129] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:39 2022] GET /identity/v3/users/f361cc03b1bf40ce81b648b897538d31 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:40.086904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c1569-445d-47e5-8914-668c75aad1ad None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:40.127775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 64/130] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:40.128663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cf6b8-c5bb-4a6d-80d1-cd21f3b5eeaf None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:40.200086 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:40.203000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c1569-445d-47e5-8914-668c75aad1ad None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:40.203575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:40.240634 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cf6b8-c5bb-4a6d-80d1-cd21f3b5eeaf None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:40.241415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722ae4d-4082-4c58-8530-bce30839b29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:40.242138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722ae4d-4082-4c58-8530-bce30839b29c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:40.242916 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722ae4d-4082-4c58-8530-bce30839b29c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:40.243254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722ae4d-4082-4c58-8530-bce30839b29c None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:40.244646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722ae4d-4082-4c58-8530-bce30839b29c None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:40.246221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722ae4d-4082-4c58-8530-bce30839b29c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:40.248089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 67/131] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:40 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:40.248958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722ae4d-4082-4c58-8530-bce30839b29c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:40.277656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cfe09-eabf-43c3-8672-5bc2faa32e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:40.278333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cfe09-eabf-43c3-8672-5bc2faa32e90 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:40.278706 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cfe09-eabf-43c3-8672-5bc2faa32e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:40.278983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cfe09-eabf-43c3-8672-5bc2faa32e90 None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce/users/8a7b708ff6c140a3ba833eb00eff7926/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:40.281043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cfe09-eabf-43c3-8672-5bc2faa32e90 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7203986da504ab1ba9f76d7f64f7dce, user_id=8a7b708ff6c140a3ba833eb00eff7926, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:40.282705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cfe09-eabf-43c3-8672-5bc2faa32e90 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:40.299847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 65/132] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:14:40 2022] PUT /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce/users/8a7b708ff6c140a3ba833eb00eff7926/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:14:40.300870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cfe09-eabf-43c3-8672-5bc2faa32e90 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:40.407731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722ae4d-4082-4c58-8530-bce30839b29c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:40.417678 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cfe09-eabf-43c3-8672-5bc2faa32e90 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:40.446178 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:40.454846 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71a09d-da6c-4d2d-a7b1-a576ae9c3ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:40.455913 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71a09d-da6c-4d2d-a7b1-a576ae9c3ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:40.456198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71a09d-da6c-4d2d-a7b1-a576ae9c3ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:40.456468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71a09d-da6c-4d2d-a7b1-a576ae9c3ab1 None admin] PATH_INFO: `/v3/projects/498b5a50ecb147c8b5c7f428dddc9c4b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:40.457683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71a09d-da6c-4d2d-a7b1-a576ae9c3ab1 None admin] RBAC: Authorizing `identity:get_project(project_id=498b5a50ecb147c8b5c7f428dddc9c4b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:40.457963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c61b0-fa67-4ebb-8a77-3c43cc32e83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:40.458833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c61b0-fa67-4ebb-8a77-3c43cc32e83a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:40.459151 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71a09d-da6c-4d2d-a7b1-a576ae9c3ab1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:40.459496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c61b0-fa67-4ebb-8a77-3c43cc32e83a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:40.459952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c61b0-fa67-4ebb-8a77-3c43cc32e83a None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:40.460967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 68/133] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:40 2022] GET /identity/v3/projects/498b5a50ecb147c8b5c7f428dddc9c4b => generated 312 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:40.461786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c61b0-fa67-4ebb-8a77-3c43cc32e83a None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:40.463617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c61b0-fa67-4ebb-8a77-3c43cc32e83a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:40.465727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71a09d-da6c-4d2d-a7b1-a576ae9c3ab1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:40.465964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 66/134] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:40 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:40.467577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c61b0-fa67-4ebb-8a77-3c43cc32e83a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:40.507789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:40.579901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71a09d-da6c-4d2d-a7b1-a576ae9c3ab1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:40.583213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c61b0-fa67-4ebb-8a77-3c43cc32e83a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:40.617878 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab54f7-d73d-4fc0-a07a-b0833b08e34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:40.618551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab54f7-d73d-4fc0-a07a-b0833b08e34e None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:40.618958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab54f7-d73d-4fc0-a07a-b0833b08e34e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:40.619208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab54f7-d73d-4fc0-a07a-b0833b08e34e None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce/users/f361cc03b1bf40ce81b648b897538d31/roles/84212031716c4cf8a5083824fe47dbb7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:40.621424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab54f7-d73d-4fc0-a07a-b0833b08e34e None admin] RBAC: Authorizing `identity:create_grant(project_id=d7203986da504ab1ba9f76d7f64f7dce, user_id=f361cc03b1bf40ce81b648b897538d31, role_id=84212031716c4cf8a5083824fe47dbb7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:40.621903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdedd2-04e6-4b99-8a2c-678584e15c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:40.622586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdedd2-04e6-4b99-8a2c-678584e15c36 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:40.622866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdedd2-04e6-4b99-8a2c-678584e15c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:40.623139 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdedd2-04e6-4b99-8a2c-678584e15c36 None admin] PATH_INFO: `/v3/projects/498b5a50ecb147c8b5c7f428dddc9c4b/users/f361cc03b1bf40ce81b648b897538d31/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:40.623960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab54f7-d73d-4fc0-a07a-b0833b08e34e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:40.625067 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdedd2-04e6-4b99-8a2c-678584e15c36 None admin] RBAC: Authorizing `identity:create_grant(project_id=498b5a50ecb147c8b5c7f428dddc9c4b, user_id=f361cc03b1bf40ce81b648b897538d31, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:40.626588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdedd2-04e6-4b99-8a2c-678584e15c36 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:40.646663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 69/135] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:14:40 2022] PUT /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce/users/f361cc03b1bf40ce81b648b897538d31/roles/84212031716c4cf8a5083824fe47dbb7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:14:40.648693 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 67/136] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:14:40 2022] PUT /identity/v3/projects/498b5a50ecb147c8b5c7f428dddc9c4b/users/f361cc03b1bf40ce81b648b897538d31/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:14:40.650467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afba47-0e79-4759-817d-a3b4dfff177c None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:40.650749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afba47-0e79-4759-817d-a3b4dfff177c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:40.651026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afba47-0e79-4759-817d-a3b4dfff177c None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:40.652553 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 68/137] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:40.653691 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab54f7-d73d-4fc0-a07a-b0833b08e34e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:40.661842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a16e1f-cc80-4d36-94d7-63a26604f9b1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:40.662470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a16e1f-cc80-4d36-94d7-63a26604f9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:40.662965 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a16e1f-cc80-4d36-94d7-63a26604f9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:40.685956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b5a16e1f-cc80-4d36-94d7-63a26604f9b1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:40.687192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b5a16e1f-cc80-4d36-94d7-63a26604f9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:40.761654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:40.777149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 69/138] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:40.855495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:40.862353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab54f7-d73d-4fc0-a07a-b0833b08e34e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:40.903861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326595c-af4e-40b4-95ad-26e803f2fc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:40.904500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326595c-af4e-40b4-95ad-26e803f2fc23 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:40.904779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326595c-af4e-40b4-95ad-26e803f2fc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:40.905051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326595c-af4e-40b4-95ad-26e803f2fc23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:40.905821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326595c-af4e-40b4-95ad-26e803f2fc23 None admin] RBAC: Adding query filter params (role.id=7dae07bb44b2421baecc0522aaf94e7b, scope.project.id=d7203986da504ab1ba9f76d7f64f7dce, user.id=a0475613e7004333a8bd6e4342c3069e) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:40.906124 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326595c-af4e-40b4-95ad-26e803f2fc23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:40.907574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326595c-af4e-40b4-95ad-26e803f2fc23 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:40.913843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 70/139] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:40 2022] GET /identity/v3/role_assignments?user.id=a0475613e7004333a8bd6e4342c3069e&scope.project.id=d7203986da504ab1ba9f76d7f64f7dce&role.id=7dae07bb44b2421baecc0522aaf94e7b => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:41.090713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a16e1f-cc80-4d36-94d7-63a26604f9b1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:41.201990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a16e1f-cc80-4d36-94d7-63a26604f9b1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:41.246228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afebee8-9cb0-4d91-9d7c-db10eacdc459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:41.246877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afebee8-9cb0-4d91-9d7c-db10eacdc459 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:41.247153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afebee8-9cb0-4d91-9d7c-db10eacdc459 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:41.247422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afebee8-9cb0-4d91-9d7c-db10eacdc459 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:41.265941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afebee8-9cb0-4d91-9d7c-db10eacdc459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-2afebee8-9cb0-4d91-9d7c-db10eacdc459 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:41.271326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:41.272441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:41.273615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 70/140] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:41.277970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326595c-af4e-40b4-95ad-26e803f2fc23 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:41.411172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326595c-af4e-40b4-95ad-26e803f2fc23 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:41.453923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3898a-1b10-4641-9562-2730c09b1902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:41.454952 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3898a-1b10-4641-9562-2730c09b1902 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:41.455435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3898a-1b10-4641-9562-2730c09b1902 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:41.455919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3898a-1b10-4641-9562-2730c09b1902 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:41.456853 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3898a-1b10-4641-9562-2730c09b1902 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:41.457377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3898a-1b10-4641-9562-2730c09b1902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:41.458972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3898a-1b10-4641-9562-2730c09b1902 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:41.472624 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 71/141] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:14:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:41.478410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afebee8-9cb0-4d91-9d7c-db10eacdc459 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:41.559948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:41.634384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afebee8-9cb0-4d91-9d7c-db10eacdc459 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:41.695816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842102a-b9fb-446c-b07e-b7aabf8faf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:41.696481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842102a-b9fb-446c-b07e-b7aabf8faf8d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:41.696754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842102a-b9fb-446c-b07e-b7aabf8faf8d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:41.697106 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842102a-b9fb-446c-b07e-b7aabf8faf8d None admin] PATH_INFO: `/v3/users/f361cc03b1bf40ce81b648b897538d31` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:41.698308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842102a-b9fb-446c-b07e-b7aabf8faf8d None admin] RBAC: Authorizing `identity:get_user(user_id=f361cc03b1bf40ce81b648b897538d31)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:41.699755 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842102a-b9fb-446c-b07e-b7aabf8faf8d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:41.701373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 71/142] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:41 2022] GET /identity/v3/users/f361cc03b1bf40ce81b648b897538d31 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:41.706991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3898a-1b10-4641-9562-2730c09b1902 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:41.750892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:41.838514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3898a-1b10-4641-9562-2730c09b1902 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:41.879682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c42a6-8fa2-462b-8e1d-070461450b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:41.880554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c42a6-8fa2-462b-8e1d-070461450b67 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:41.880831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c42a6-8fa2-462b-8e1d-070461450b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:41.881152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c42a6-8fa2-462b-8e1d-070461450b67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:41.958205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c42a6-8fa2-462b-8e1d-070461450b67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:41.960009 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c42a6-8fa2-462b-8e1d-070461450b67 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:41.991688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:42.071280 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 72/143] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:42.077112 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842102a-b9fb-446c-b07e-b7aabf8faf8d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:42.207340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842102a-b9fb-446c-b07e-b7aabf8faf8d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:42.221147 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:42.231650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d3ba0-2219-4a7c-9e1f-1f5e9041d968 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:42.231938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d3ba0-2219-4a7c-9e1f-1f5e9041d968 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:42.232194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d3ba0-2219-4a7c-9e1f-1f5e9041d968 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:42.234314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 73/144] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:14:42.249096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56129895-1152-46b2-8486-9a9215267d9f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:42.249433 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56129895-1152-46b2-8486-9a9215267d9f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:42.249760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56129895-1152-46b2-8486-9a9215267d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:42.253603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a8ff1-0149-481c-acf8-ee7638e3fabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:42.254276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a8ff1-0149-481c-acf8-ee7638e3fabc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:42.254569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a8ff1-0149-481c-acf8-ee7638e3fabc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:42.254910 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a8ff1-0149-481c-acf8-ee7638e3fabc None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:42.259249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a8ff1-0149-481c-acf8-ee7638e3fabc None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:42.261123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a8ff1-0149-481c-acf8-ee7638e3fabc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:42.265325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 72/145] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:42 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:42.275780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a8ff1-0149-481c-acf8-ee7638e3fabc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:42.281730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-56129895-1152-46b2-8486-9a9215267d9f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:42.282959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-56129895-1152-46b2-8486-9a9215267d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:42.380703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 74/146] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:42.464756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a8ff1-0149-481c-acf8-ee7638e3fabc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:42.511732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce03312-7d7f-469b-8c52-a060b5cea862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:42.512537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce03312-7d7f-469b-8c52-a060b5cea862 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:42.512537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce03312-7d7f-469b-8c52-a060b5cea862 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:42.512900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce03312-7d7f-469b-8c52-a060b5cea862 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:42.513513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce03312-7d7f-469b-8c52-a060b5cea862 None admin] RBAC: Adding query filter params (role.id=231a681c35614ea3aaf3452a8c167c5c, scope.project.id=d7203986da504ab1ba9f76d7f64f7dce, user.id=f361cc03b1bf40ce81b648b897538d31) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:42.513817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce03312-7d7f-469b-8c52-a060b5cea862 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:42.515247 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce03312-7d7f-469b-8c52-a060b5cea862 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:42.525611 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 73/147] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:42 2022] GET /identity/v3/role_assignments?user.id=f361cc03b1bf40ce81b648b897538d31&scope.project.id=d7203986da504ab1ba9f76d7f64f7dce&role.id=231a681c35614ea3aaf3452a8c167c5c => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:42.806161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56129895-1152-46b2-8486-9a9215267d9f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:42.876384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:42.930804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56129895-1152-46b2-8486-9a9215267d9f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:42.968625 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8395a7-4469-4d72-986b-faa35cb174c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:42.969307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8395a7-4469-4d72-986b-faa35cb174c4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:42.969586 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8395a7-4469-4d72-986b-faa35cb174c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:42.969855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8395a7-4469-4d72-986b-faa35cb174c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:42.970978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8395a7-4469-4d72-986b-faa35cb174c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:42.972425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8395a7-4469-4d72-986b-faa35cb174c4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:42.974246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 75/148] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:42.979730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce03312-7d7f-469b-8c52-a060b5cea862 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:43.030061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:43.085307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce03312-7d7f-469b-8c52-a060b5cea862 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:43.119606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a0b98-ec29-4598-a950-ac849393ffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:43.120225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a0b98-ec29-4598-a950-ac849393ffee None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:43.120507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a0b98-ec29-4598-a950-ac849393ffee None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:43.120833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a0b98-ec29-4598-a950-ac849393ffee None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:43.121530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a0b98-ec29-4598-a950-ac849393ffee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:43.123077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a0b98-ec29-4598-a950-ac849393ffee None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:43.161016 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 74/149] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:14:42 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:14:43.617895 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872846f-1348-407d-b63c-d9903cd89b05 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:43.618305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872846f-1348-407d-b63c-d9903cd89b05 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:43.618471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872846f-1348-407d-b63c-d9903cd89b05 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:43.619932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 76/150] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:43.625866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56696581-7b65-4123-9316-90558417940f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:43.626800 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56696581-7b65-4123-9316-90558417940f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:43.627313 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56696581-7b65-4123-9316-90558417940f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:43.653863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-56696581-7b65-4123-9316-90558417940f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:43.656177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-56696581-7b65-4123-9316-90558417940f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:43.743596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 75/151] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:43.769237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f024b5-21ce-42eb-a9c6-69cac4d7d3d7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:43.769560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f024b5-21ce-42eb-a9c6-69cac4d7d3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:43.769793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f024b5-21ce-42eb-a9c6-69cac4d7d3d7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:43.771318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 77/152] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:43.776564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cb16a-2e16-48f2-b541-05a3884e2d0e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:43.776918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cb16a-2e16-48f2-b541-05a3884e2d0e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:43.777224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cb16a-2e16-48f2-b541-05a3884e2d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:43.795892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-504cb16a-2e16-48f2-b541-05a3884e2d0e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:43.798944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-504cb16a-2e16-48f2-b541-05a3884e2d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:43.825901 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:43.878797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 76/153] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:43.969770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52448fb-492f-479d-8f0c-c81ad24426b3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:43.970727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52448fb-492f-479d-8f0c-c81ad24426b3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:43.971214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52448fb-492f-479d-8f0c-c81ad24426b3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:43.972747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 78/154] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:43.978777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af15880-8813-419e-9680-d442f3eb42f1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:43.979284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af15880-8813-419e-9680-d442f3eb42f1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:43.979827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af15880-8813-419e-9680-d442f3eb42f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:44.000980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-8af15880-8813-419e-9680-d442f3eb42f1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:44.002505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8af15880-8813-419e-9680-d442f3eb42f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:44.036365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52448fb-492f-479d-8f0c-c81ad24426b3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:44.079552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 77/155] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:44.168172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52448fb-492f-479d-8f0c-c81ad24426b3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:44.191779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:44.200414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af15880-8813-419e-9680-d442f3eb42f1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:44.209025 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74672ccd-3306-4a61-a34a-f37ad4da49e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:44.209705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74672ccd-3306-4a61-a34a-f37ad4da49e2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:44.210031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74672ccd-3306-4a61-a34a-f37ad4da49e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:44.210303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74672ccd-3306-4a61-a34a-f37ad4da49e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:44.221624 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74672ccd-3306-4a61-a34a-f37ad4da49e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-74672ccd-3306-4a61-a34a-f37ad4da49e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:44.226707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:44.227563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:14:44.228574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 79/156] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:14:44.233808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74672ccd-3306-4a61-a34a-f37ad4da49e2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:44.319228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:44.328655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af15880-8813-419e-9680-d442f3eb42f1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:44.344816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74672ccd-3306-4a61-a34a-f37ad4da49e2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:44.374080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50157f07-ce7d-44d2-ac79-c25b2bd3312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:44.375194 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50157f07-ce7d-44d2-ac79-c25b2bd3312a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:44.375670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50157f07-ce7d-44d2-ac79-c25b2bd3312a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:44.376198 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50157f07-ce7d-44d2-ac79-c25b2bd3312a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:44.380743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acada07-da12-4d67-ac6f-be65582756a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:44.381392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acada07-da12-4d67-ac6f-be65582756a3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:44.381666 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acada07-da12-4d67-ac6f-be65582756a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:44.381982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acada07-da12-4d67-ac6f-be65582756a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:44.382651 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acada07-da12-4d67-ac6f-be65582756a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:44.383105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acada07-da12-4d67-ac6f-be65582756a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:44.384724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acada07-da12-4d67-ac6f-be65582756a3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:44.388904 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50157f07-ce7d-44d2-ac79-c25b2bd3312a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:44.393605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 80/157] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:44.394873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379358a5-edf7-41f2-a73e-c4c84c826ac4 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:44.395159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379358a5-edf7-41f2-a73e-c4c84c826ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-50157f07-ce7d-44d2-ac79-c25b2bd3312a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:44.395378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:44.396281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:44.397037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:44.397037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:44.397037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:44.397037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:44.397037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:44.397037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:44.397037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:44.397037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:44.397037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:44.397037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:44.397037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:44.397037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:44.397655 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379358a5-edf7-41f2-a73e-c4c84c826ac4 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:44.397719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 78/158] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:14:44.397827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 81/159] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:44.397898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50157f07-ce7d-44d2-ac79-c25b2bd3312a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:44.403905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91530a-2aa5-4460-a11d-8dae5c2d37ae None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:44.404247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91530a-2aa5-4460-a11d-8dae5c2d37ae None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:44.404582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91530a-2aa5-4460-a11d-8dae5c2d37ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:44.423713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2e91530a-2aa5-4460-a11d-8dae5c2d37ae None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:44.424922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2e91530a-2aa5-4460-a11d-8dae5c2d37ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:44.467760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:44.507754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 82/160] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:14:44.508832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91530a-2aa5-4460-a11d-8dae5c2d37ae None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:44.523863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50157f07-ce7d-44d2-ac79-c25b2bd3312a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:44.571881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63eb35-3a1a-4893-b046-2a3b2ef87673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:44.572791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63eb35-3a1a-4893-b046-2a3b2ef87673 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:44.573555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63eb35-3a1a-4893-b046-2a3b2ef87673 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:44.573970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63eb35-3a1a-4893-b046-2a3b2ef87673 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:44.589026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63eb35-3a1a-4893-b046-2a3b2ef87673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-5b63eb35-3a1a-4893-b046-2a3b2ef87673 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:44.595922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:44.597117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:44.598142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:44.598142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:44.598142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:44.598142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:44.598142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:44.598142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:44.598142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:44.598142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:44.598142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:44.598142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:44.598142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:14:44.598142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:44.600770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 79/161] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:14:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:44.601927 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63eb35-3a1a-4893-b046-2a3b2ef87673 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:44.620532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:44.650768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91530a-2aa5-4460-a11d-8dae5c2d37ae None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:44.700912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38260726-a8b1-46dc-86e1-edcf92cc72fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:44.701855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38260726-a8b1-46dc-86e1-edcf92cc72fa None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:44.702191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38260726-a8b1-46dc-86e1-edcf92cc72fa None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:44.702467 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38260726-a8b1-46dc-86e1-edcf92cc72fa None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:44.703197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38260726-a8b1-46dc-86e1-edcf92cc72fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:44.703503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38260726-a8b1-46dc-86e1-edcf92cc72fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:44.704898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38260726-a8b1-46dc-86e1-edcf92cc72fa None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:44.714322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 83/162] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:14:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:44.714567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38260726-a8b1-46dc-86e1-edcf92cc72fa None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:44.735403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63eb35-3a1a-4893-b046-2a3b2ef87673 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:44.757503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:44.802786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c8a83-e19c-4f9f-aa79-91ff2434ca35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:44.803734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c8a83-e19c-4f9f-aa79-91ff2434ca35 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:44.804212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c8a83-e19c-4f9f-aa79-91ff2434ca35 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:44.804658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c8a83-e19c-4f9f-aa79-91ff2434ca35 None admin] PATH_INFO: `/v3/users/8a7b708ff6c140a3ba833eb00eff7926` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:44.806270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c8a83-e19c-4f9f-aa79-91ff2434ca35 None admin] RBAC: Authorizing `identity:get_user(user_id=8a7b708ff6c140a3ba833eb00eff7926)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:44.807924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c8a83-e19c-4f9f-aa79-91ff2434ca35 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:44.809937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 80/163] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:44 2022] GET /identity/v3/users/8a7b708ff6c140a3ba833eb00eff7926 => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:44.818625 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c8a83-e19c-4f9f-aa79-91ff2434ca35 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:44.824971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38260726-a8b1-46dc-86e1-edcf92cc72fa None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:44.859369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc0a61-e47a-4568-97d1-9a7c3ed83fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:44.859987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc0a61-e47a-4568-97d1-9a7c3ed83fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:44.860265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc0a61-e47a-4568-97d1-9a7c3ed83fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:44.860544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc0a61-e47a-4568-97d1-9a7c3ed83fd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:44.861243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc0a61-e47a-4568-97d1-9a7c3ed83fd7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:44.861548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc0a61-e47a-4568-97d1-9a7c3ed83fd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:44.862991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc0a61-e47a-4568-97d1-9a7c3ed83fd7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:44.879729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 84/164] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:14:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:44.880599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc0a61-e47a-4568-97d1-9a7c3ed83fd7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:44.935094 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:44.974845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c8a83-e19c-4f9f-aa79-91ff2434ca35 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:45.021311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ec624-d916-4bac-b41e-f1ab0634ec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:45.021962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ec624-d916-4bac-b41e-f1ab0634ec93 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:45.022241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ec624-d916-4bac-b41e-f1ab0634ec93 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:45.022512 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ec624-d916-4bac-b41e-f1ab0634ec93 None admin] PATH_INFO: `/v3/users/f361cc03b1bf40ce81b648b897538d31` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:45.024166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ec624-d916-4bac-b41e-f1ab0634ec93 None admin] RBAC: Authorizing `identity:get_user(user_id=f361cc03b1bf40ce81b648b897538d31)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:45.025786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ec624-d916-4bac-b41e-f1ab0634ec93 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:45.027435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 81/165] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:44 2022] GET /identity/v3/users/f361cc03b1bf40ce81b648b897538d31 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:45.028212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ec624-d916-4bac-b41e-f1ab0634ec93 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:45.082458 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:45.103272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc0a61-e47a-4568-97d1-9a7c3ed83fd7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:45.163698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ec624-d916-4bac-b41e-f1ab0634ec93 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:45.167388 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a4935-7430-43df-81b4-792861e2ba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:45.168250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a4935-7430-43df-81b4-792861e2ba92 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:45.168739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a4935-7430-43df-81b4-792861e2ba92 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:45.169246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a4935-7430-43df-81b4-792861e2ba92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:45.216976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532fb5c-d421-42cb-8cb3-9b0c299f24f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:45.218169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532fb5c-d421-42cb-8cb3-9b0c299f24f9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:45.218683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532fb5c-d421-42cb-8cb3-9b0c299f24f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:45.219143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532fb5c-d421-42cb-8cb3-9b0c299f24f9 None admin] PATH_INFO: `/v3/users/f361cc03b1bf40ce81b648b897538d31` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:45.220504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532fb5c-d421-42cb-8cb3-9b0c299f24f9 None admin] RBAC: Authorizing `identity:get_user(user_id=f361cc03b1bf40ce81b648b897538d31)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:45.222513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532fb5c-d421-42cb-8cb3-9b0c299f24f9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:45.224799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 82/166] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:45 2022] GET /identity/v3/users/f361cc03b1bf40ce81b648b897538d31 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:45.225929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532fb5c-d421-42cb-8cb3-9b0c299f24f9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:45.243238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:45.309719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a4935-7430-43df-81b4-792861e2ba92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:45.312382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a4935-7430-43df-81b4-792861e2ba92 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:45.391370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532fb5c-d421-42cb-8cb3-9b0c299f24f9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:45.457232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6a27d-f61f-4362-9f6b-aa898db18176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:45.458614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6a27d-f61f-4362-9f6b-aa898db18176 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:45.459088 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6a27d-f61f-4362-9f6b-aa898db18176 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:45.459548 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6a27d-f61f-4362-9f6b-aa898db18176 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:45.474032 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6a27d-f61f-4362-9f6b-aa898db18176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e5c6a27d-f61f-4362-9f6b-aa898db18176 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:45.479929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:45.480861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:45.481689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:45.481689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:45.481689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:45.481689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:45.481689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:45.481689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:45.481689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:45.481689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:45.481689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:45.481689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:45.481689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:14:45.481689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:45.484441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 83/167] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:45.485479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6a27d-f61f-4362-9f6b-aa898db18176 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:45.506408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:45.528082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 85/168] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:45.528947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a4935-7430-43df-81b4-792861e2ba92 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:45.581937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:45.611984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6a27d-f61f-4362-9f6b-aa898db18176 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:45.658132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae90614-af1f-4dd0-931f-7f2176ff29fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:45.658816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae90614-af1f-4dd0-931f-7f2176ff29fd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:45.659027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae90614-af1f-4dd0-931f-7f2176ff29fd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:45.659300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae90614-af1f-4dd0-931f-7f2176ff29fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:45.761385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a4935-7430-43df-81b4-792861e2ba92 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:45.786231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae90614-af1f-4dd0-931f-7f2176ff29fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:45.787714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae90614-af1f-4dd0-931f-7f2176ff29fd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:45.799766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45221b46-60d6-40b4-980d-8f03e66029fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:45.800508 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45221b46-60d6-40b4-980d-8f03e66029fc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:45.800853 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45221b46-60d6-40b4-980d-8f03e66029fc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:45.801221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45221b46-60d6-40b4-980d-8f03e66029fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:45.865106 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:45.873107 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45221b46-60d6-40b4-980d-8f03e66029fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:45.875565 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45221b46-60d6-40b4-980d-8f03e66029fc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:45.958452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:45.982981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 86/169] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:45.983933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45221b46-60d6-40b4-980d-8f03e66029fc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:46.000074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 84/170] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:46.000941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae90614-af1f-4dd0-931f-7f2176ff29fd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:46.107857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae90614-af1f-4dd0-931f-7f2176ff29fd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:46.124175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:46.132281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45221b46-60d6-40b4-980d-8f03e66029fc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:46.151135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161199f-3584-4149-b60a-b64082f221fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:46.152069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161199f-3584-4149-b60a-b64082f221fb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:46.152440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161199f-3584-4149-b60a-b64082f221fb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:46.152800 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161199f-3584-4149-b60a-b64082f221fb None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:46.161904 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161199f-3584-4149-b60a-b64082f221fb None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:46.163372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161199f-3584-4149-b60a-b64082f221fb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:46.166077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 85/171] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:46 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:46.166876 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161199f-3584-4149-b60a-b64082f221fb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:46.214369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3074d-1389-4b2f-8f01-763aaab20aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:46.215233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3074d-1389-4b2f-8f01-763aaab20aff None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:46.215513 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3074d-1389-4b2f-8f01-763aaab20aff None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:46.215781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3074d-1389-4b2f-8f01-763aaab20aff None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:46.216518 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3074d-1389-4b2f-8f01-763aaab20aff None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:46.216820 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3074d-1389-4b2f-8f01-763aaab20aff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:46.224336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3074d-1389-4b2f-8f01-763aaab20aff None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:46.243523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 87/172] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:14:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:46.244241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3074d-1389-4b2f-8f01-763aaab20aff None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:46.267742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:46.353309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161199f-3584-4149-b60a-b64082f221fb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:46.360650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3074d-1389-4b2f-8f01-763aaab20aff None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:46.401788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d25b9-2d56-4257-aa6c-eb9c7652ef6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:46.402434 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d25b9-2d56-4257-aa6c-eb9c7652ef6d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:46.402708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d25b9-2d56-4257-aa6c-eb9c7652ef6d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:46.402977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d25b9-2d56-4257-aa6c-eb9c7652ef6d None admin] PATH_INFO: `/v3/projects/498b5a50ecb147c8b5c7f428dddc9c4b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:46.404551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d25b9-2d56-4257-aa6c-eb9c7652ef6d None admin] RBAC: Authorizing `identity:get_project(project_id=498b5a50ecb147c8b5c7f428dddc9c4b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:46.406022 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d25b9-2d56-4257-aa6c-eb9c7652ef6d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:46.407655 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 88/173] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:46 2022] GET /identity/v3/projects/498b5a50ecb147c8b5c7f428dddc9c4b => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:46.408419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d25b9-2d56-4257-aa6c-eb9c7652ef6d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:46.427619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba427666-08a9-4e1e-bb33-bb0393c1effe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:46.428266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba427666-08a9-4e1e-bb33-bb0393c1effe None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:46.428547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba427666-08a9-4e1e-bb33-bb0393c1effe None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:46.428814 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba427666-08a9-4e1e-bb33-bb0393c1effe None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:46.429969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba427666-08a9-4e1e-bb33-bb0393c1effe None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:46.433144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba427666-08a9-4e1e-bb33-bb0393c1effe None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:46.436652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 86/174] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:46 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:46.437453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba427666-08a9-4e1e-bb33-bb0393c1effe None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:46.466583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:46.629026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d25b9-2d56-4257-aa6c-eb9c7652ef6d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:46.651060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba427666-08a9-4e1e-bb33-bb0393c1effe None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:46.675675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:46.682408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6817b-a82f-49a0-8ca8-45e084497ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:46.683062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6817b-a82f-49a0-8ca8-45e084497ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:46.683338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6817b-a82f-49a0-8ca8-45e084497ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:46.683607 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6817b-a82f-49a0-8ca8-45e084497ff3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:46.684344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6817b-a82f-49a0-8ca8-45e084497ff3 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=d7203986da504ab1ba9f76d7f64f7dce, user.id=8a7b708ff6c140a3ba833eb00eff7926) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:46.684647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6817b-a82f-49a0-8ca8-45e084497ff3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:46.686132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6817b-a82f-49a0-8ca8-45e084497ff3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:46.692480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 89/175] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:46 2022] GET /identity/v3/role_assignments?user.id=8a7b708ff6c140a3ba833eb00eff7926&scope.project.id=d7203986da504ab1ba9f76d7f64f7dce&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:46.693328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6817b-a82f-49a0-8ca8-45e084497ff3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:46.740781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91a806-3e3f-4fdf-9e06-ddf327c1a61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:46.741486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a806-3e3f-4fdf-9e06-ddf327c1a61b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:46.741766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a806-3e3f-4fdf-9e06-ddf327c1a61b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:46.742035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a806-3e3f-4fdf-9e06-ddf327c1a61b None admin] PATH_INFO: `/v3/users/a0475613e7004333a8bd6e4342c3069e` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:46.745959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91a806-3e3f-4fdf-9e06-ddf327c1a61b None admin] RBAC: Authorizing `identity:get_user(user_id=a0475613e7004333a8bd6e4342c3069e)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:46.747791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91a806-3e3f-4fdf-9e06-ddf327c1a61b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:46.750163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 87/176] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:46 2022] GET /identity/v3/users/a0475613e7004333a8bd6e4342c3069e => generated 289 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:46.751246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91a806-3e3f-4fdf-9e06-ddf327c1a61b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:46.809211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:46.820003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6817b-a82f-49a0-8ca8-45e084497ff3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:46.857475 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759617d7-d70f-431d-a529-8ec742969636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:46.858413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759617d7-d70f-431d-a529-8ec742969636 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:46.858832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759617d7-d70f-431d-a529-8ec742969636 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:46.859252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759617d7-d70f-431d-a529-8ec742969636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:46.860142 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759617d7-d70f-431d-a529-8ec742969636 None admin] RBAC: Adding query filter params (role.id=231a681c35614ea3aaf3452a8c167c5c, scope.project.id=498b5a50ecb147c8b5c7f428dddc9c4b, user.id=f361cc03b1bf40ce81b648b897538d31) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:46.860612 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759617d7-d70f-431d-a529-8ec742969636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:46.862314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759617d7-d70f-431d-a529-8ec742969636 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:46.869554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 90/177] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:46 2022] GET /identity/v3/role_assignments?user.id=f361cc03b1bf40ce81b648b897538d31&scope.project.id=498b5a50ecb147c8b5c7f428dddc9c4b&role.id=231a681c35614ea3aaf3452a8c167c5c => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:46.871101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9b650-d606-4c6b-a2d1-a832d3ce6e92 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:46.871547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9b650-d606-4c6b-a2d1-a832d3ce6e92 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:46.871962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9b650-d606-4c6b-a2d1-a832d3ce6e92 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:46.873577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 91/178] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:46.874652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9b650-d606-4c6b-a2d1-a832d3ce6e92 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:46.979431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91a806-3e3f-4fdf-9e06-ddf327c1a61b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:47.058394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8f031-965e-4fbc-8674-ce1e70fdec97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:47.059040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8f031-965e-4fbc-8674-ce1e70fdec97 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:47.059319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8f031-965e-4fbc-8674-ce1e70fdec97 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:47.059586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8f031-965e-4fbc-8674-ce1e70fdec97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:47.060575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8f031-965e-4fbc-8674-ce1e70fdec97 None admin] RBAC: Adding query filter params (role.id=84212031716c4cf8a5083824fe47dbb7, scope.project.id=d7203986da504ab1ba9f76d7f64f7dce, user.id=f361cc03b1bf40ce81b648b897538d31) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:47.062217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8f031-965e-4fbc-8674-ce1e70fdec97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:47.063827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8f031-965e-4fbc-8674-ce1e70fdec97 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:47.079434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 88/179] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:46 2022] GET /identity/v3/role_assignments?user.id=f361cc03b1bf40ce81b648b897538d31&scope.project.id=d7203986da504ab1ba9f76d7f64f7dce&role.id=84212031716c4cf8a5083824fe47dbb7 => generated 608 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:47.080769 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c72f3-b56c-48fe-9c3a-e32d3b134f5e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:47.081054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c72f3-b56c-48fe-9c3a-e32d3b134f5e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:47.081355 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c72f3-b56c-48fe-9c3a-e32d3b134f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:47.106058 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9b650-d606-4c6b-a2d1-a832d3ce6e92 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:47.123807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f43c72f3-b56c-48fe-9c3a-e32d3b134f5e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:47.125752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f43c72f3-b56c-48fe-9c3a-e32d3b134f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:47.169312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac4a0f-c70d-4c79-85aa-b159089165b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:47.170046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac4a0f-c70d-4c79-85aa-b159089165b5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:47.170234 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac4a0f-c70d-4c79-85aa-b159089165b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:47.170511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac4a0f-c70d-4c79-85aa-b159089165b5 None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:47.173045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac4a0f-c70d-4c79-85aa-b159089165b5 None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:47.174742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:47.175603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac4a0f-c70d-4c79-85aa-b159089165b5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:47.181289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 92/180] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:46 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:47.183089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecb571-648b-4d8b-8c33-4bb764f39b44 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:47.183595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecb571-648b-4d8b-8c33-4bb764f39b44 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:47.184131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecb571-648b-4d8b-8c33-4bb764f39b44 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:47.186655 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 93/181] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:47.194267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ecb571-648b-4d8b-8c33-4bb764f39b44 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:47.226658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 89/182] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:47.226929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56a00a-c6ad-4cc5-94f6-70a71f1ea68a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:47.227052 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56a00a-c6ad-4cc5-94f6-70a71f1ea68a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:47.227418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56a00a-c6ad-4cc5-94f6-70a71f1ea68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:47.244474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:47.252422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-df56a00a-c6ad-4cc5-94f6-70a71f1ea68a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:47.254155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-df56a00a-c6ad-4cc5-94f6-70a71f1ea68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:47.343974 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 90/183] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:47.345437 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff95bd7-8705-4df2-998a-d12b722ba347 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:47.345738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff95bd7-8705-4df2-998a-d12b722ba347 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:47.346100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff95bd7-8705-4df2-998a-d12b722ba347 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:47.347459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 91/184] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:47.353066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670a762-c9a2-4c91-b05d-290b58e624a5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:47.353382 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670a762-c9a2-4c91-b05d-290b58e624a5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:47.353758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670a762-c9a2-4c91-b05d-290b58e624a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:47.373680 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d670a762-c9a2-4c91-b05d-290b58e624a5 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:47.374164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ecb571-648b-4d8b-8c33-4bb764f39b44 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:47.374856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d670a762-c9a2-4c91-b05d-290b58e624a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:47.421065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc74c85-cfa2-4ab4-8158-67bbc330151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:47.421928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc74c85-cfa2-4ab4-8158-67bbc330151d None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:47.422208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc74c85-cfa2-4ab4-8158-67bbc330151d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:47.422483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc74c85-cfa2-4ab4-8158-67bbc330151d None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce/users/a0475613e7004333a8bd6e4342c3069e/roles/7dae07bb44b2421baecc0522aaf94e7b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:47.425492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc74c85-cfa2-4ab4-8158-67bbc330151d None admin] RBAC: Authorizing `identity:create_grant(project_id=d7203986da504ab1ba9f76d7f64f7dce, user_id=a0475613e7004333a8bd6e4342c3069e, role_id=7dae07bb44b2421baecc0522aaf94e7b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:47.427774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc74c85-cfa2-4ab4-8158-67bbc330151d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:47.443087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 94/185] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:14:47 2022] PUT /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce/users/a0475613e7004333a8bd6e4342c3069e/roles/7dae07bb44b2421baecc0522aaf94e7b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:14:47.463639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 92/186] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:14:47.573734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc74c85-cfa2-4ab4-8158-67bbc330151d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:47.673121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:47.674166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670a762-c9a2-4c91-b05d-290b58e624a5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:47.693994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc74c85-cfa2-4ab4-8158-67bbc330151d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:47.717028 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:47.732870 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64032ced-1e42-4f71-89cc-94a3fe198348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:47.733553 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64032ced-1e42-4f71-89cc-94a3fe198348 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:47.733830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64032ced-1e42-4f71-89cc-94a3fe198348 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:47.734100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64032ced-1e42-4f71-89cc-94a3fe198348 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:47.745741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64032ced-1e42-4f71-89cc-94a3fe198348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-64032ced-1e42-4f71-89cc-94a3fe198348 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:47.751320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:47.752199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:14:47.753044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 95/187] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:14:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:47.758160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64032ced-1e42-4f71-89cc-94a3fe198348 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:47.818261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670a762-c9a2-4c91-b05d-290b58e624a5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:47.854115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fed62-01f9-446b-b30b-aa909e1a8b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:47.854680 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fed62-01f9-446b-b30b-aa909e1a8b47 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:47.854963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fed62-01f9-446b-b30b-aa909e1a8b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:47.855216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fed62-01f9-446b-b30b-aa909e1a8b47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:47.856467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0fed62-01f9-446b-b30b-aa909e1a8b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:47.857935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0fed62-01f9-446b-b30b-aa909e1a8b47 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:47.861323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 93/188] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:47.861323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fed62-01f9-446b-b30b-aa909e1a8b47 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:47.863637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64032ced-1e42-4f71-89cc-94a3fe198348 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:47.898980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4867748-fe58-4a62-8200-de40ba0178c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:47.899590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4867748-fe58-4a62-8200-de40ba0178c0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:47.899987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4867748-fe58-4a62-8200-de40ba0178c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:47.900264 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4867748-fe58-4a62-8200-de40ba0178c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:47.901031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4867748-fe58-4a62-8200-de40ba0178c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:47.901364 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4867748-fe58-4a62-8200-de40ba0178c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:47.902744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4867748-fe58-4a62-8200-de40ba0178c0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:47.916371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 96/189] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:47.916371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4867748-fe58-4a62-8200-de40ba0178c0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:47.919646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:47.966563 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:47.969493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fed62-01f9-446b-b30b-aa909e1a8b47 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:48.005643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4d89c-6f51-4090-bae3-2231f983fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:48.006273 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4d89c-6f51-4090-bae3-2231f983fd96 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:48.006555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4d89c-6f51-4090-bae3-2231f983fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:48.006825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4d89c-6f51-4090-bae3-2231f983fd96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:48.017820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4d89c-6f51-4090-bae3-2231f983fd96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:48.022656 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4867748-fe58-4a62-8200-de40ba0178c0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e0a4d89c-6f51-4090-bae3-2231f983fd96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:48.022767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:48.023600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 94/190] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:14:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:48.029433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4d89c-6f51-4090-bae3-2231f983fd96 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:48.062551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd82b6-cdaa-448e-849d-1df1844c8c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:48.063509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd82b6-cdaa-448e-849d-1df1844c8c1b None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:48.064016 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd82b6-cdaa-448e-849d-1df1844c8c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:48.064863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd82b6-cdaa-448e-849d-1df1844c8c1b None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:48.065756 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd82b6-cdaa-448e-849d-1df1844c8c1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:48.067749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd82b6-cdaa-448e-849d-1df1844c8c1b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:48.090423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 97/191] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:14:47 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:14:48.091674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd82b6-cdaa-448e-849d-1df1844c8c1b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:48.134827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4d89c-6f51-4090-bae3-2231f983fd96 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:48.173815 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3340a7-9547-4fa0-a1e3-83d71727e19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:48.174469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3340a7-9547-4fa0-a1e3-83d71727e19a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:48.174745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3340a7-9547-4fa0-a1e3-83d71727e19a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:48.175063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3340a7-9547-4fa0-a1e3-83d71727e19a None admin] PATH_INFO: `/v3/users/96a9af3f683345c393c32d2d15c05551` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:48.187342 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:48.201328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3340a7-9547-4fa0-a1e3-83d71727e19a None admin] RBAC: Authorizing `identity:get_user(user_id=96a9af3f683345c393c32d2d15c05551)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:48.203839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3340a7-9547-4fa0-a1e3-83d71727e19a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:48.205688 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 95/192] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:48 2022] GET /identity/v3/users/96a9af3f683345c393c32d2d15c05551 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:48.211414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3340a7-9547-4fa0-a1e3-83d71727e19a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:48.231968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:48.262713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd82b6-cdaa-448e-849d-1df1844c8c1b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:48.306966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832254a-5538-48d6-86f4-32f622b18a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:48.307694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832254a-5538-48d6-86f4-32f622b18a77 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:48.307874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832254a-5538-48d6-86f4-32f622b18a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:48.308147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832254a-5538-48d6-86f4-32f622b18a77 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:48.308881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832254a-5538-48d6-86f4-32f622b18a77 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:48.309310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832254a-5538-48d6-86f4-32f622b18a77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:48.310718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832254a-5538-48d6-86f4-32f622b18a77 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:48.321368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 98/193] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:14:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:48.325917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832254a-5538-48d6-86f4-32f622b18a77 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:48.343233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3340a7-9547-4fa0-a1e3-83d71727e19a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:48.389386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d840b1-0eb6-4d87-a844-d2287aff1519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:48.390342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d840b1-0eb6-4d87-a844-d2287aff1519 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:48.391480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d840b1-0eb6-4d87-a844-d2287aff1519 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:48.391959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d840b1-0eb6-4d87-a844-d2287aff1519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:48.447575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832254a-5538-48d6-86f4-32f622b18a77 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:48.477383 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:48.479142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d840b1-0eb6-4d87-a844-d2287aff1519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:48.480951 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d840b1-0eb6-4d87-a844-d2287aff1519 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:48.492731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd0ad8-a48b-494f-aea1-613f986a0924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:48.493821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd0ad8-a48b-494f-aea1-613f986a0924 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:48.494319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd0ad8-a48b-494f-aea1-613f986a0924 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:48.494780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd0ad8-a48b-494f-aea1-613f986a0924 None admin] PATH_INFO: `/v3/users/96a9af3f683345c393c32d2d15c05551` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:48.497747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd0ad8-a48b-494f-aea1-613f986a0924 None admin] RBAC: Authorizing `identity:get_user(user_id=96a9af3f683345c393c32d2d15c05551)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:48.499461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd0ad8-a48b-494f-aea1-613f986a0924 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:48.500560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:48.501751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 99/194] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:48 2022] GET /identity/v3/users/96a9af3f683345c393c32d2d15c05551 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:48.508294 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd0ad8-a48b-494f-aea1-613f986a0924 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:48.593222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 96/195] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 04:14:48.598793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d840b1-0eb6-4d87-a844-d2287aff1519 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:48.630545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd0ad8-a48b-494f-aea1-613f986a0924 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:48.675433 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ee306-ed18-49ee-bd6f-42ba6301750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:48.676161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ee306-ed18-49ee-bd6f-42ba6301750b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:48.676436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ee306-ed18-49ee-bd6f-42ba6301750b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:48.676702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ee306-ed18-49ee-bd6f-42ba6301750b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:48.707274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d840b1-0eb6-4d87-a844-d2287aff1519 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:48.726263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:48.741023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:48.746097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb252846-492e-417c-87fb-827b91455da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:48.747180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252846-492e-417c-87fb-827b91455da8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:48.747536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252846-492e-417c-87fb-827b91455da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:48.748139 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252846-492e-417c-87fb-827b91455da8 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:48.749435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb252846-492e-417c-87fb-827b91455da8 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:48.751120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb252846-492e-417c-87fb-827b91455da8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:48.753715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 97/196] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:48 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:48.754230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ee306-ed18-49ee-bd6f-42ba6301750b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:48.755887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ee306-ed18-49ee-bd6f-42ba6301750b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:48.760909 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb252846-492e-417c-87fb-827b91455da8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:48.879230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb252846-492e-417c-87fb-827b91455da8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:48.889947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 100/197] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 04:14:48.895505 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ee306-ed18-49ee-bd6f-42ba6301750b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:48.918084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18f5bc-6613-48cd-bcfd-65fa9d23d7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:48.919026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18f5bc-6613-48cd-bcfd-65fa9d23d7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:48.919469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18f5bc-6613-48cd-bcfd-65fa9d23d7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:48.919903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18f5bc-6613-48cd-bcfd-65fa9d23d7b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:48.920858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18f5bc-6613-48cd-bcfd-65fa9d23d7b0 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=96874534e5534d3699fb67c58215c104, user.id=96a9af3f683345c393c32d2d15c05551) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:48.921439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18f5bc-6613-48cd-bcfd-65fa9d23d7b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:48.923077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18f5bc-6613-48cd-bcfd-65fa9d23d7b0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:48.929235 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 98/198] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:48 2022] GET /identity/v3/role_assignments?user.id=96a9af3f683345c393c32d2d15c05551&scope.project.id=96874534e5534d3699fb67c58215c104&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:49.017731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:49.028355 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ee306-ed18-49ee-bd6f-42ba6301750b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:49.064729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0f280-62de-422b-adf9-abad5dca6d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:49.065401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0f280-62de-422b-adf9-abad5dca6d9e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:49.065678 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0f280-62de-422b-adf9-abad5dca6d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:49.065949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0f280-62de-422b-adf9-abad5dca6d9e None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:49.067166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0f280-62de-422b-adf9-abad5dca6d9e None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:49.068653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0f280-62de-422b-adf9-abad5dca6d9e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:49.070511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 101/199] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:48 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:49.076581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18f5bc-6613-48cd-bcfd-65fa9d23d7b0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:49.172519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:49.243055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18f5bc-6613-48cd-bcfd-65fa9d23d7b0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:49.276566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0af9a-38b7-4f59-ac0e-4861e53099f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:49.277207 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0af9a-38b7-4f59-ac0e-4861e53099f1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:49.277488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0af9a-38b7-4f59-ac0e-4861e53099f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:49.277756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0af9a-38b7-4f59-ac0e-4861e53099f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:49.278491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0af9a-38b7-4f59-ac0e-4861e53099f1 None admin] RBAC: Adding query filter params (role.id=84212031716c4cf8a5083824fe47dbb7, scope.project.id=96874534e5534d3699fb67c58215c104, user.id=96a9af3f683345c393c32d2d15c05551) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:49.278798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0af9a-38b7-4f59-ac0e-4861e53099f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:49.280239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0af9a-38b7-4f59-ac0e-4861e53099f1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:49.286298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 99/200] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:49 2022] GET /identity/v3/role_assignments?user.id=96a9af3f683345c393c32d2d15c05551&scope.project.id=96874534e5534d3699fb67c58215c104&role.id=84212031716c4cf8a5083824fe47dbb7 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:49.982783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25912c0a-0a9b-4604-bdfd-749849c89d80 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:49.983296 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25912c0a-0a9b-4604-bdfd-749849c89d80 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:49.983296 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25912c0a-0a9b-4604-bdfd-749849c89d80 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:49.984829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 102/201] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:50.000018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5412c-fb33-42e8-b678-8b006a592c9a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:50.000303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5412c-fb33-42e8-b678-8b006a592c9a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:50.000574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5412c-fb33-42e8-b678-8b006a592c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:50.020272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e7f5412c-fb33-42e8-b678-8b006a592c9a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:50.021514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e7f5412c-fb33-42e8-b678-8b006a592c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:50.091859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 100/202] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:50.558634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25912c0a-0a9b-4604-bdfd-749849c89d80 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:50.709164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25912c0a-0a9b-4604-bdfd-749849c89d80 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:50.751564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed6e84-7bf0-48af-bdb4-9a5acc313e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:50.752531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed6e84-7bf0-48af-bdb4-9a5acc313e36 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:50.752971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed6e84-7bf0-48af-bdb4-9a5acc313e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:50.753428 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed6e84-7bf0-48af-bdb4-9a5acc313e36 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:50.774204 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed6e84-7bf0-48af-bdb4-9a5acc313e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d3ed6e84-7bf0-48af-bdb4-9a5acc313e36 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:50.779411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:50.780263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:50.781052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:50.781052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:50.781052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:50.781052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:50.781052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:50.781052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:50.781052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:50.781052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:50.781052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:50.781052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:50.781052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:14:50.781052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:14:50.782945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 103/203] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:50.789350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5412c-fb33-42e8-b678-8b006a592c9a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:50.849599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:50.896894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822ee6e-ebe3-4142-b5fa-6efc16c70caa None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:50.897569 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822ee6e-ebe3-4142-b5fa-6efc16c70caa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:50.898029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822ee6e-ebe3-4142-b5fa-6efc16c70caa None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:50.899766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 104/204] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:50.906608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18298ad3-9a40-4739-a534-9dc9be1dbb6a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:50.907157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18298ad3-9a40-4739-a534-9dc9be1dbb6a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:50.907612 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18298ad3-9a40-4739-a534-9dc9be1dbb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:50.916537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5412c-fb33-42e8-b678-8b006a592c9a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:50.974220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-18298ad3-9a40-4739-a534-9dc9be1dbb6a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:50.975543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-18298ad3-9a40-4739-a534-9dc9be1dbb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:50.985912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da40a2-3078-4f1b-9a2e-f7237baf9e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:50.989110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da40a2-3078-4f1b-9a2e-f7237baf9e29 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:50.989110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da40a2-3078-4f1b-9a2e-f7237baf9e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:50.989110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da40a2-3078-4f1b-9a2e-f7237baf9e29 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:50.989110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da40a2-3078-4f1b-9a2e-f7237baf9e29 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:50.989110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da40a2-3078-4f1b-9a2e-f7237baf9e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:50.989656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da40a2-3078-4f1b-9a2e-f7237baf9e29 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:51.005761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 101/205] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:14:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:51.011974 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da40a2-3078-4f1b-9a2e-f7237baf9e29 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:51.019322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:51.169107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da40a2-3078-4f1b-9a2e-f7237baf9e29 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:51.180775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:51.202983 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 105/206] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:51.234567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90f8cc-8935-452e-ba5d-4f9bffbd6fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:51.235211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90f8cc-8935-452e-ba5d-4f9bffbd6fce None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:51.235488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90f8cc-8935-452e-ba5d-4f9bffbd6fce None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:51.237669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90f8cc-8935-452e-ba5d-4f9bffbd6fce None admin] PATH_INFO: `/v3/users/a0475613e7004333a8bd6e4342c3069e` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:51.239421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90f8cc-8935-452e-ba5d-4f9bffbd6fce None admin] RBAC: Authorizing `identity:get_user(user_id=a0475613e7004333a8bd6e4342c3069e)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:51.241694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90f8cc-8935-452e-ba5d-4f9bffbd6fce None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:51.243919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 102/207] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:51 2022] GET /identity/v3/users/a0475613e7004333a8bd6e4342c3069e => generated 289 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:51.254226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18298ad3-9a40-4739-a534-9dc9be1dbb6a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:51.298643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca028b-82e4-4b7d-819f-6ae00adaddd1 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:51.298989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca028b-82e4-4b7d-819f-6ae00adaddd1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:51.299193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca028b-82e4-4b7d-819f-6ae00adaddd1 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:51.301069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 103/208] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:51.306631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee363f0d-e0cd-4b92-b113-38c20822cf13 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:51.306936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee363f0d-e0cd-4b92-b113-38c20822cf13 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:51.307295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee363f0d-e0cd-4b92-b113-38c20822cf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:51.364492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ee363f0d-e0cd-4b92-b113-38c20822cf13 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:51.367918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ee363f0d-e0cd-4b92-b113-38c20822cf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:51.515875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18298ad3-9a40-4739-a534-9dc9be1dbb6a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:51.559153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6bad1-676b-40ba-8913-46f2b0b876ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:51.559782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6bad1-676b-40ba-8913-46f2b0b876ad None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:51.560060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6bad1-676b-40ba-8913-46f2b0b876ad None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:51.560331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6bad1-676b-40ba-8913-46f2b0b876ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:51.561143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 104/209] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:51.562009 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee363f0d-e0cd-4b92-b113-38c20822cf13 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:51.618125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:51.682599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee363f0d-e0cd-4b92-b113-38c20822cf13 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:51.699132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:51.712150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6bad1-676b-40ba-8913-46f2b0b876ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:51.713737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6bad1-676b-40ba-8913-46f2b0b876ad None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:51.735875 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49507309-3ab6-4c68-b889-98ac543825ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:51.736927 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49507309-3ab6-4c68-b889-98ac543825ad None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:51.737517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49507309-3ab6-4c68-b889-98ac543825ad None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:51.737968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49507309-3ab6-4c68-b889-98ac543825ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:51.749265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49507309-3ab6-4c68-b889-98ac543825ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-49507309-3ab6-4c68-b889-98ac543825ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:51.754722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:51.755586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:51.756386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:51.756386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:51.756386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:51.756386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:51.756386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:51.756386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:51.756386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:51.756386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:51.756386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:51.756386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:51.756386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:51.756386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:51.758726 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 105/210] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:51.760197 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4e27b-ba57-4baf-8bbc-c2f60e28ed12 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:51.760778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4e27b-ba57-4baf-8bbc-c2f60e28ed12 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:51.761348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4e27b-ba57-4baf-8bbc-c2f60e28ed12 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:51.762891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 106/211] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:51.768281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56c503-f13d-4828-91b0-d875536c0f5c None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:51.768784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56c503-f13d-4828-91b0-d875536c0f5c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:51.769279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56c503-f13d-4828-91b0-d875536c0f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:51.788050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-8d56c503-f13d-4828-91b0-d875536c0f5c None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:51.789711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8d56c503-f13d-4828-91b0-d875536c0f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:51.841232 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 106/212] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:51.845486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6bad1-676b-40ba-8913-46f2b0b876ad None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:51.874660 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 107/213] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:51.877326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56c503-f13d-4828-91b0-d875536c0f5c None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:51.913251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:51.968691 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6bad1-676b-40ba-8913-46f2b0b876ad None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:51.993179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:52.000841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56c503-f13d-4828-91b0-d875536c0f5c None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:52.003698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d60c3-2b32-4f9a-86a8-28ff95ff1dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:52.004010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d60c3-2b32-4f9a-86a8-28ff95ff1dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:52.004279 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d60c3-2b32-4f9a-86a8-28ff95ff1dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:52.004543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d60c3-2b32-4f9a-86a8-28ff95ff1dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:52.005230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d60c3-2b32-4f9a-86a8-28ff95ff1dd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:52.005526 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d60c3-2b32-4f9a-86a8-28ff95ff1dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:52.006899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d60c3-2b32-4f9a-86a8-28ff95ff1dd3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:52.015268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 107/214] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:14:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:52.016041 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d60c3-2b32-4f9a-86a8-28ff95ff1dd3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:52.040747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd527bc-422b-4664-9a1d-3e4ef56a0670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:52.041419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd527bc-422b-4664-9a1d-3e4ef56a0670 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:52.041698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd527bc-422b-4664-9a1d-3e4ef56a0670 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:52.041960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd527bc-422b-4664-9a1d-3e4ef56a0670 None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:52.043083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd527bc-422b-4664-9a1d-3e4ef56a0670 None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:52.044519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd527bc-422b-4664-9a1d-3e4ef56a0670 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:52.046199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 108/215] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:51 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:52.047434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b2ffe-136a-46f2-9eb6-a1ca4e1b4d30 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:52.047719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b2ffe-136a-46f2-9eb6-a1ca4e1b4d30 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:52.047994 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b2ffe-136a-46f2-9eb6-a1ca4e1b4d30 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:52.053242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 109/216] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:52.053242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b2ffe-136a-46f2-9eb6-a1ca4e1b4d30 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:52.119749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d60c3-2b32-4f9a-86a8-28ff95ff1dd3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:52.130253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:52.151883 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b2ffe-136a-46f2-9eb6-a1ca4e1b4d30 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:52.155945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b2b7b-11d5-4985-b9b8-06a04c86736a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:52.156573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b2b7b-11d5-4985-b9b8-06a04c86736a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:52.156843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b2b7b-11d5-4985-b9b8-06a04c86736a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:52.157129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b2b7b-11d5-4985-b9b8-06a04c86736a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:52.158224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b2b7b-11d5-4985-b9b8-06a04c86736a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:52.159723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b2b7b-11d5-4985-b9b8-06a04c86736a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:52.161324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 108/217] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:52.162134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b2b7b-11d5-4985-b9b8-06a04c86736a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:52.186026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b608d1e-e5ae-4996-8aac-919caab2a7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:52.186667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b608d1e-e5ae-4996-8aac-919caab2a7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:52.186942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b608d1e-e5ae-4996-8aac-919caab2a7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:52.187213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b608d1e-e5ae-4996-8aac-919caab2a7b9 None admin] PATH_INFO: `/v3/users/b4917bf683594d41b65e7ac042229ea6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:52.200871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b608d1e-e5ae-4996-8aac-919caab2a7b9 None admin] RBAC: Authorizing `identity:get_user(user_id=b4917bf683594d41b65e7ac042229ea6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:52.202387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b608d1e-e5ae-4996-8aac-919caab2a7b9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:52.204017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 110/218] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:52 2022] GET /identity/v3/users/b4917bf683594d41b65e7ac042229ea6 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:52.205314 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48505b-4ffa-4446-9fc3-7df98c17f4d7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:52.205593 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48505b-4ffa-4446-9fc3-7df98c17f4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:52.205865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48505b-4ffa-4446-9fc3-7df98c17f4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:52.222486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6a48505b-4ffa-4446-9fc3-7df98c17f4d7 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:52.223657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6a48505b-4ffa-4446-9fc3-7df98c17f4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:52.237480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:52.262093 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b2b7b-11d5-4985-b9b8-06a04c86736a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:52.292149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 111/219] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:52.292907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48505b-4ffa-4446-9fc3-7df98c17f4d7 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:52.294210 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e85ce-eddc-48b0-b409-b8329cdc1f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:52.294818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e85ce-eddc-48b0-b409-b8329cdc1f98 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:52.295095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e85ce-eddc-48b0-b409-b8329cdc1f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:52.295362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e85ce-eddc-48b0-b409-b8329cdc1f98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:52.296111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e85ce-eddc-48b0-b409-b8329cdc1f98 None admin] RBAC: Adding query filter params (role.id=7dae07bb44b2421baecc0522aaf94e7b, scope.project.id=d7203986da504ab1ba9f76d7f64f7dce, user.id=a0475613e7004333a8bd6e4342c3069e) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:52.296416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e85ce-eddc-48b0-b409-b8329cdc1f98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:52.297881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e85ce-eddc-48b0-b409-b8329cdc1f98 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:52.304851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 109/220] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:52 2022] GET /identity/v3/role_assignments?user.id=a0475613e7004333a8bd6e4342c3069e&scope.project.id=d7203986da504ab1ba9f76d7f64f7dce&role.id=7dae07bb44b2421baecc0522aaf94e7b => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:52.305635 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e85ce-eddc-48b0-b409-b8329cdc1f98 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:52.374711 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:52.433053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48505b-4ffa-4446-9fc3-7df98c17f4d7 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:52.458510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e85ce-eddc-48b0-b409-b8329cdc1f98 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:52.471807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e44203-a919-40c1-8a88-80215cfe1b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:52.472825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e44203-a919-40c1-8a88-80215cfe1b5e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:52.473334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e44203-a919-40c1-8a88-80215cfe1b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:52.473808 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e44203-a919-40c1-8a88-80215cfe1b5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:52.485329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e44203-a919-40c1-8a88-80215cfe1b5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d2e44203-a919-40c1-8a88-80215cfe1b5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:52.491199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:52.492090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:52.493270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 112/221] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:14:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:52.498448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e44203-a919-40c1-8a88-80215cfe1b5e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:52.511124 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05042c-d20c-4b6d-9e10-89512bfe59be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:52.511747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05042c-d20c-4b6d-9e10-89512bfe59be None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:52.512025 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05042c-d20c-4b6d-9e10-89512bfe59be None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:52.512321 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:52.512426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05042c-d20c-4b6d-9e10-89512bfe59be None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:52.513394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05042c-d20c-4b6d-9e10-89512bfe59be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:52.515742 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05042c-d20c-4b6d-9e10-89512bfe59be None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:52.543005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 110/222] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:14:52 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:14:52.543788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05042c-d20c-4b6d-9e10-89512bfe59be None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:52.629367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e44203-a919-40c1-8a88-80215cfe1b5e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:52.669496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b2352-f20f-4829-b8ee-eac16fecc7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:52.670124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b2352-f20f-4829-b8ee-eac16fecc7dc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:52.670405 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b2352-f20f-4829-b8ee-eac16fecc7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:52.670671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b2352-f20f-4829-b8ee-eac16fecc7dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:52.690680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:52.695827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05042c-d20c-4b6d-9e10-89512bfe59be None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:52.735211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484009a-8c80-436f-b02d-a6a0b46ad9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:52.735211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484009a-8c80-436f-b02d-a6a0b46ad9df None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:52.735598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484009a-8c80-436f-b02d-a6a0b46ad9df None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:52.735937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484009a-8c80-436f-b02d-a6a0b46ad9df None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:52.736621 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484009a-8c80-436f-b02d-a6a0b46ad9df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:52.736926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484009a-8c80-436f-b02d-a6a0b46ad9df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:52.738377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484009a-8c80-436f-b02d-a6a0b46ad9df None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:52.746000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b2352-f20f-4829-b8ee-eac16fecc7dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:52.747463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b2352-f20f-4829-b8ee-eac16fecc7dc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:52.750138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 111/223] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:52.750138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484009a-8c80-436f-b02d-a6a0b46ad9df None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:52.760111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:52.876652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 113/224] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:52.877270 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484009a-8c80-436f-b02d-a6a0b46ad9df None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:52.877455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b2352-f20f-4829-b8ee-eac16fecc7dc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:52.919987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49e567-36f8-4102-9ce2-c50544a600f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:52.921104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49e567-36f8-4102-9ce2-c50544a600f3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:52.921381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49e567-36f8-4102-9ce2-c50544a600f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:52.921655 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49e567-36f8-4102-9ce2-c50544a600f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:52.936965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49e567-36f8-4102-9ce2-c50544a600f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8b49e567-36f8-4102-9ce2-c50544a600f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:52.942344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:52.943265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:14:52.944051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 112/225] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:14:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:52.944707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49e567-36f8-4102-9ce2-c50544a600f3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:52.975333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:53.000765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b2352-f20f-4829-b8ee-eac16fecc7dc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:53.012749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:53.040825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507fcbb-c264-4fd5-9541-cbc58aa749a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:53.041920 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507fcbb-c264-4fd5-9541-cbc58aa749a4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:53.042433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507fcbb-c264-4fd5-9541-cbc58aa749a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:53.042893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507fcbb-c264-4fd5-9541-cbc58aa749a4 None admin] PATH_INFO: `/v3/users/96a9af3f683345c393c32d2d15c05551` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:53.044260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507fcbb-c264-4fd5-9541-cbc58aa749a4 None admin] RBAC: Authorizing `identity:get_user(user_id=96a9af3f683345c393c32d2d15c05551)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:53.046036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507fcbb-c264-4fd5-9541-cbc58aa749a4 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:53.048029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 114/226] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:52 2022] GET /identity/v3/users/96a9af3f683345c393c32d2d15c05551 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:53.049230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507fcbb-c264-4fd5-9541-cbc58aa749a4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:53.075208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49e567-36f8-4102-9ce2-c50544a600f3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:53.114595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c3feb-1229-46f0-b439-619e5aebb513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:53.116058 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c3feb-1229-46f0-b439-619e5aebb513 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:53.116633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c3feb-1229-46f0-b439-619e5aebb513 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:53.117112 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c3feb-1229-46f0-b439-619e5aebb513 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:53.119268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c3feb-1229-46f0-b439-619e5aebb513 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:53.121345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c3feb-1229-46f0-b439-619e5aebb513 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:53.124226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 113/227] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:52 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:53.125389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c3feb-1229-46f0-b439-619e5aebb513 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:53.168030 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507fcbb-c264-4fd5-9541-cbc58aa749a4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:53.213108 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4360e8-f84f-4735-abcd-e7c3fc8886e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:53.213750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4360e8-f84f-4735-abcd-e7c3fc8886e4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:53.214092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4360e8-f84f-4735-abcd-e7c3fc8886e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:53.214421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4360e8-f84f-4735-abcd-e7c3fc8886e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:53.215149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4360e8-f84f-4735-abcd-e7c3fc8886e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:53.215512 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4360e8-f84f-4735-abcd-e7c3fc8886e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:53.216973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4360e8-f84f-4735-abcd-e7c3fc8886e4 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:53.230513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 115/228] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:14:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:53.231434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4360e8-f84f-4735-abcd-e7c3fc8886e4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:53.234056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:53.250154 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c3feb-1229-46f0-b439-619e5aebb513 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:53.281489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:53.293165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73adfc-6d88-4753-8e93-ac9aac02af91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:53.294298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73adfc-6d88-4753-8e93-ac9aac02af91 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:53.294778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73adfc-6d88-4753-8e93-ac9aac02af91 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:53.295338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73adfc-6d88-4753-8e93-ac9aac02af91 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:53.297343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73adfc-6d88-4753-8e93-ac9aac02af91 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:53.299762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73adfc-6d88-4753-8e93-ac9aac02af91 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:53.301611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 114/229] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:53 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:53.302873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73adfc-6d88-4753-8e93-ac9aac02af91 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:53.346199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4360e8-f84f-4735-abcd-e7c3fc8886e4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:53.380684 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f8028-6965-4c67-b648-59af91e6516c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:53.381486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f8028-6965-4c67-b648-59af91e6516c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:53.381857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f8028-6965-4c67-b648-59af91e6516c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:53.382246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f8028-6965-4c67-b648-59af91e6516c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:53.383079 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f8028-6965-4c67-b648-59af91e6516c None admin] RBAC: Adding query filter params (role.id=231a681c35614ea3aaf3452a8c167c5c, scope.project.id=96874534e5534d3699fb67c58215c104, user.id=b4917bf683594d41b65e7ac042229ea6) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:53.383485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f8028-6965-4c67-b648-59af91e6516c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:53.385068 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f8028-6965-4c67-b648-59af91e6516c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:53.391261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 116/230] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:53 2022] GET /identity/v3/role_assignments?user.id=b4917bf683594d41b65e7ac042229ea6&scope.project.id=96874534e5534d3699fb67c58215c104&role.id=231a681c35614ea3aaf3452a8c167c5c => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:53.392223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f8028-6965-4c67-b648-59af91e6516c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:53.420849 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73adfc-6d88-4753-8e93-ac9aac02af91 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:53.459430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f411f5d-4517-47cc-8a0a-e19fd761e7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:53.460477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f411f5d-4517-47cc-8a0a-e19fd761e7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:53.460956 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f411f5d-4517-47cc-8a0a-e19fd761e7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:53.461478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f411f5d-4517-47cc-8a0a-e19fd761e7f2 None admin] PATH_INFO: `/v3/users/96a9af3f683345c393c32d2d15c05551` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:53.463067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f411f5d-4517-47cc-8a0a-e19fd761e7f2 None admin] RBAC: Authorizing `identity:get_user(user_id=96a9af3f683345c393c32d2d15c05551)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:53.464786 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f411f5d-4517-47cc-8a0a-e19fd761e7f2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:53.467070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 115/231] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:53 2022] GET /identity/v3/users/96a9af3f683345c393c32d2d15c05551 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:53.472237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f411f5d-4517-47cc-8a0a-e19fd761e7f2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:53.530247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:53.540154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f8028-6965-4c67-b648-59af91e6516c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:53.547725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:53.621365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c7ddd-c4c6-4d77-8501-2fee1a5711b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:53.624262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c7ddd-c4c6-4d77-8501-2fee1a5711b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:53.624536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c7ddd-c4c6-4d77-8501-2fee1a5711b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:53.624802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c7ddd-c4c6-4d77-8501-2fee1a5711b9 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104/users/96a9af3f683345c393c32d2d15c05551/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:53.627870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c7ddd-c4c6-4d77-8501-2fee1a5711b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=96874534e5534d3699fb67c58215c104, user_id=96a9af3f683345c393c32d2d15c05551, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:53.629376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c7ddd-c4c6-4d77-8501-2fee1a5711b9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:53.656211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 117/232] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:14:53 2022] PUT /identity/v3/projects/96874534e5534d3699fb67c58215c104/users/96a9af3f683345c393c32d2d15c05551/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:14:53.693822 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f411f5d-4517-47cc-8a0a-e19fd761e7f2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:53.736520 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07486e-a013-4015-be6a-431993349ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:53.737232 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07486e-a013-4015-be6a-431993349ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:53.737563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07486e-a013-4015-be6a-431993349ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:53.737832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07486e-a013-4015-be6a-431993349ed8 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:53.738970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07486e-a013-4015-be6a-431993349ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:53.740519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07486e-a013-4015-be6a-431993349ed8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:53.742925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 116/233] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:53 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:53.748453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c7ddd-c4c6-4d77-8501-2fee1a5711b9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:53.891399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c7ddd-c4c6-4d77-8501-2fee1a5711b9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:53.926795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb36c27-a671-42b5-9f0b-07615dba47c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:53.927469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb36c27-a671-42b5-9f0b-07615dba47c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:53.927733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb36c27-a671-42b5-9f0b-07615dba47c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:53.928012 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb36c27-a671-42b5-9f0b-07615dba47c6 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104/users/96a9af3f683345c393c32d2d15c05551/roles/84212031716c4cf8a5083824fe47dbb7` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:53.930207 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb36c27-a671-42b5-9f0b-07615dba47c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=96874534e5534d3699fb67c58215c104, user_id=96a9af3f683345c393c32d2d15c05551, role_id=84212031716c4cf8a5083824fe47dbb7)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:53.931684 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb36c27-a671-42b5-9f0b-07615dba47c6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:53.951572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 118/234] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:14:53 2022] PUT /identity/v3/projects/96874534e5534d3699fb67c58215c104/users/96a9af3f683345c393c32d2d15c05551/roles/84212031716c4cf8a5083824fe47dbb7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:14:55.133779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c74c7-7606-41d4-9360-58fb4be7c6f5 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:55.134365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c74c7-7606-41d4-9360-58fb4be7c6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:55.134365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c74c7-7606-41d4-9360-58fb4be7c6f5 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:55.135966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 117/235] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:55.141115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d3d6d-1666-4bbc-a3f8-128a3e1396f1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:55.141518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d3d6d-1666-4bbc-a3f8-128a3e1396f1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:55.141703 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d3d6d-1666-4bbc-a3f8-128a3e1396f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:55.160471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a44d3d6d-1666-4bbc-a3f8-128a3e1396f1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:55.161704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a44d3d6d-1666-4bbc-a3f8-128a3e1396f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:55.179802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:55.230472 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 119/236] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:55.258025 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098f1da-6610-4a20-98ff-d6ccb392576f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:55.258469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098f1da-6610-4a20-98ff-d6ccb392576f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:55.258834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098f1da-6610-4a20-98ff-d6ccb392576f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:55.260551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 118/237] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:55.266254 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5ffda-b80e-4393-a6c0-d1fa9587793a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:55.266727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5ffda-b80e-4393-a6c0-d1fa9587793a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:55.267142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5ffda-b80e-4393-a6c0-d1fa9587793a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:55.294027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-03e5ffda-b80e-4393-a6c0-d1fa9587793a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:55.295436 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-03e5ffda-b80e-4393-a6c0-d1fa9587793a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:55.391835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 120/238] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:55.508039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098f1da-6610-4a20-98ff-d6ccb392576f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:55.602407 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:55.611786 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098f1da-6610-4a20-98ff-d6ccb392576f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:55.645982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cacaf7-71cf-4901-bc3f-0fb5463e1bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:55.646534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cacaf7-71cf-4901-bc3f-0fb5463e1bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:55.646806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cacaf7-71cf-4901-bc3f-0fb5463e1bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:55.647066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cacaf7-71cf-4901-bc3f-0fb5463e1bf0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:55.657372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cacaf7-71cf-4901-bc3f-0fb5463e1bf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-75cacaf7-71cf-4901-bc3f-0fb5463e1bf0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:55.662217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:55.663110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:14:55.664113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 119/239] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:55.671279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5ffda-b80e-4393-a6c0-d1fa9587793a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:55.786363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:55.794471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cacaf7-71cf-4901-bc3f-0fb5463e1bf0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:55.802766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5ffda-b80e-4393-a6c0-d1fa9587793a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:55.838867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10d7d8-c368-4c52-8cf4-6e0acfcf0dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:55.839953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10d7d8-c368-4c52-8cf4-6e0acfcf0dfc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:55.840268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10d7d8-c368-4c52-8cf4-6e0acfcf0dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:55.840575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10d7d8-c368-4c52-8cf4-6e0acfcf0dfc None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:55.841318 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10d7d8-c368-4c52-8cf4-6e0acfcf0dfc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:55.841592 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10d7d8-c368-4c52-8cf4-6e0acfcf0dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:55.842968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10d7d8-c368-4c52-8cf4-6e0acfcf0dfc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:55.852193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 121/240] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:14:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:55.857689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10d7d8-c368-4c52-8cf4-6e0acfcf0dfc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:55.933906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cacaf7-71cf-4901-bc3f-0fb5463e1bf0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:55.998073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf10bb-443f-484d-8dec-cfd9723defd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:55.999171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf10bb-443f-484d-8dec-cfd9723defd5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:55.999907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf10bb-443f-484d-8dec-cfd9723defd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:56.002018 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf10bb-443f-484d-8dec-cfd9723defd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:56.003371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf10bb-443f-484d-8dec-cfd9723defd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:56.005148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf10bb-443f-484d-8dec-cfd9723defd5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:56.008397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 120/241] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:14:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:56.021207 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf10bb-443f-484d-8dec-cfd9723defd5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:56.048392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10d7d8-c368-4c52-8cf4-6e0acfcf0dfc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:56.056314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:56.085432 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d47f6e-43bb-4410-b50d-6cb8dc4ac71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:56.086222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d47f6e-43bb-4410-b50d-6cb8dc4ac71f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:56.086603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d47f6e-43bb-4410-b50d-6cb8dc4ac71f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:56.086972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d47f6e-43bb-4410-b50d-6cb8dc4ac71f None admin] PATH_INFO: `/v3/users/d69c2706136b483cb68f173575f16333` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:56.101911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d47f6e-43bb-4410-b50d-6cb8dc4ac71f None admin] RBAC: Authorizing `identity:get_user(user_id=d69c2706136b483cb68f173575f16333)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:56.103763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d47f6e-43bb-4410-b50d-6cb8dc4ac71f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:56.106658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 122/242] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:55 2022] GET /identity/v3/users/d69c2706136b483cb68f173575f16333 => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:56.108261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608e1be-18e5-4756-9a86-1a69b91d3ea0 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:56.108614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608e1be-18e5-4756-9a86-1a69b91d3ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:56.108956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608e1be-18e5-4756-9a86-1a69b91d3ea0 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:56.111491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 123/243] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:56.112213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608e1be-18e5-4756-9a86-1a69b91d3ea0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:56.121823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:56.146741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf10bb-443f-484d-8dec-cfd9723defd5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:56.191684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff128a25-ab3f-4c96-84f6-340cfa83f062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:56.192218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff128a25-ab3f-4c96-84f6-340cfa83f062 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:56.192496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff128a25-ab3f-4c96-84f6-340cfa83f062 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:56.192765 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff128a25-ab3f-4c96-84f6-340cfa83f062 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:56.193606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff128a25-ab3f-4c96-84f6-340cfa83f062 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:56.195647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff128a25-ab3f-4c96-84f6-340cfa83f062 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:56.227110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 121/244] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:14:56 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:14:56.227110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59965b9-3ad9-4a36-adeb-9084fb27dc20 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:56.227110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59965b9-3ad9-4a36-adeb-9084fb27dc20 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:56.227110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59965b9-3ad9-4a36-adeb-9084fb27dc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:56.256985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608e1be-18e5-4756-9a86-1a69b91d3ea0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:56.265932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f59965b9-3ad9-4a36-adeb-9084fb27dc20 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:56.267282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f59965b9-3ad9-4a36-adeb-9084fb27dc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:56.302069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397173e3-e936-4d36-883b-8716664e2362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:56.303462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397173e3-e936-4d36-883b-8716664e2362 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:56.304827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397173e3-e936-4d36-883b-8716664e2362 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:56.304827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397173e3-e936-4d36-883b-8716664e2362 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:56.395634 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397173e3-e936-4d36-883b-8716664e2362 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:56.397518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397173e3-e936-4d36-883b-8716664e2362 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:56.402025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:56.478898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 122/245] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:56.505975 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aea1af-2e74-48b3-a8cb-4500610361fd None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:56.506396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aea1af-2e74-48b3-a8cb-4500610361fd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:56.506763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aea1af-2e74-48b3-a8cb-4500610361fd None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:56.508662 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 123/246] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:14:56.513657 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c95374-71d2-42c8-bce0-0732925a77e6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:56.514035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c95374-71d2-42c8-bce0-0732925a77e6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:56.514401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c95374-71d2-42c8-bce0-0732925a77e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:56.519439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 124/247] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:56.525714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397173e3-e936-4d36-883b-8716664e2362 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:56.536521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:56.547182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e6c95374-71d2-42c8-bce0-0732925a77e6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:56.549053 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e6c95374-71d2-42c8-bce0-0732925a77e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:56.635046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 124/248] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:56.652518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397173e3-e936-4d36-883b-8716664e2362 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:56.655091 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:56.689505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde46ef0-128c-4018-89ba-655f8b6e9c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:56.690145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde46ef0-128c-4018-89ba-655f8b6e9c2f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:56.690424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde46ef0-128c-4018-89ba-655f8b6e9c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:56.690694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde46ef0-128c-4018-89ba-655f8b6e9c2f None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:56.691820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde46ef0-128c-4018-89ba-655f8b6e9c2f None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:56.693425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde46ef0-128c-4018-89ba-655f8b6e9c2f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:56.695136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 125/249] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:56 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:56.701151 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c95374-71d2-42c8-bce0-0732925a77e6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:56.716619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bded9f-bdc2-4d2e-8f92-5478984f5bfe None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:56.716902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bded9f-bdc2-4d2e-8f92-5478984f5bfe None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:56.717453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bded9f-bdc2-4d2e-8f92-5478984f5bfe None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:56.718871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 126/250] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:14:56.723755 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb839ecf-2ea5-413c-a947-8f8a88fface3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:56.724031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb839ecf-2ea5-413c-a947-8f8a88fface3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:56.724302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb839ecf-2ea5-413c-a947-8f8a88fface3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:56.746516 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-bb839ecf-2ea5-413c-a947-8f8a88fface3 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:56.747636 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bb839ecf-2ea5-413c-a947-8f8a88fface3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:56.826794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c95374-71d2-42c8-bce0-0732925a77e6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:56.840167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 127/251] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:14:56.841519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb839ecf-2ea5-413c-a947-8f8a88fface3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:56.888244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8bdfc-bf8e-4f13-99f3-f1ff53d9dd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:56.889802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8bdfc-bf8e-4f13-99f3-f1ff53d9dd13 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:56.890283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8bdfc-bf8e-4f13-99f3-f1ff53d9dd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:56.890726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8bdfc-bf8e-4f13-99f3-f1ff53d9dd13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:56.892076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8bdfc-bf8e-4f13-99f3-f1ff53d9dd13 None admin] RBAC: Adding query filter params (role.id=7dae07bb44b2421baecc0522aaf94e7b, scope.project.id=96874534e5534d3699fb67c58215c104, user.id=d69c2706136b483cb68f173575f16333) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:56.892963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8bdfc-bf8e-4f13-99f3-f1ff53d9dd13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:56.894924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8bdfc-bf8e-4f13-99f3-f1ff53d9dd13 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:56.899344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:56.903088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 125/252] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:56 2022] GET /identity/v3/role_assignments?user.id=d69c2706136b483cb68f173575f16333&scope.project.id=96874534e5534d3699fb67c58215c104&role.id=7dae07bb44b2421baecc0522aaf94e7b => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:56.923359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8bdfc-bf8e-4f13-99f3-f1ff53d9dd13 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:56.980114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb839ecf-2ea5-413c-a947-8f8a88fface3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:56.981944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:57.025203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1173987-98ae-4415-bb26-501850640523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:57.025203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1173987-98ae-4415-bb26-501850640523 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:57.025518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1173987-98ae-4415-bb26-501850640523 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:57.025613 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1173987-98ae-4415-bb26-501850640523 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:57.039963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1173987-98ae-4415-bb26-501850640523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c1173987-98ae-4415-bb26-501850640523 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:57.046003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:57.046941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:57.047759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 128/253] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:57.058702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1173987-98ae-4415-bb26-501850640523 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:57.066509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8bdfc-bf8e-4f13-99f3-f1ff53d9dd13 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:57.106341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a6884-b1d6-46bb-9f11-fe89bb88f199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:57.107089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a6884-b1d6-46bb-9f11-fe89bb88f199 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:57.107367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a6884-b1d6-46bb-9f11-fe89bb88f199 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:57.107700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a6884-b1d6-46bb-9f11-fe89bb88f199 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:57.122144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a6884-b1d6-46bb-9f11-fe89bb88f199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3f3a6884-b1d6-46bb-9f11-fe89bb88f199 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:57.129181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:57.130137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:14:57.130932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 126/254] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:14:57.137721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a6884-b1d6-46bb-9f11-fe89bb88f199 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:57.198838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1173987-98ae-4415-bb26-501850640523 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:57.223562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:57.257671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6219a0-d8ab-42c0-81ee-46b73afabbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:57.258391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6219a0-d8ab-42c0-81ee-46b73afabbdf None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:57.258592 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6219a0-d8ab-42c0-81ee-46b73afabbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:57.258821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6219a0-d8ab-42c0-81ee-46b73afabbdf None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:57.259527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6219a0-d8ab-42c0-81ee-46b73afabbdf None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:57.259829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6219a0-d8ab-42c0-81ee-46b73afabbdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:57.261243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6219a0-d8ab-42c0-81ee-46b73afabbdf None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:57.270227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 129/255] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:14:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:57.271135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6219a0-d8ab-42c0-81ee-46b73afabbdf None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:57.299755 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:57.313397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a6884-b1d6-46bb-9f11-fe89bb88f199 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:57.386741 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6219a0-d8ab-42c0-81ee-46b73afabbdf None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:57.407312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff49513-7697-4191-83e2-416886856bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:57.407930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff49513-7697-4191-83e2-416886856bdd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:57.408199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff49513-7697-4191-83e2-416886856bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:57.408464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff49513-7697-4191-83e2-416886856bdd None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:57.409153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff49513-7697-4191-83e2-416886856bdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:57.409453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff49513-7697-4191-83e2-416886856bdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:57.410838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff49513-7697-4191-83e2-416886856bdd None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:57.426740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 127/256] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:57.427598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff49513-7697-4191-83e2-416886856bdd None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:57.455908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f408d4-485f-4fcd-b698-4d0d01b32a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:57.456560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f408d4-485f-4fcd-b698-4d0d01b32a74 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:57.456836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f408d4-485f-4fcd-b698-4d0d01b32a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:57.457122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f408d4-485f-4fcd-b698-4d0d01b32a74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:57.477407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f408d4-485f-4fcd-b698-4d0d01b32a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-22f408d4-485f-4fcd-b698-4d0d01b32a74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:14:57.486285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:14:57.487177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:14:57.487969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:14:57.487969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:14:57.487969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:14:57.487969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:14:57.487969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:14:57.487969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:14:57.487969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:14:57.487969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:14:57.487969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:14:57.487969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:14:57.487969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:14:57.487969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:14:57.489054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 130/257] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:14:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:14:57.491231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f408d4-485f-4fcd-b698-4d0d01b32a74 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:57.567615 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff49513-7697-4191-83e2-416886856bdd None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:57.586768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:57.590972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:57.603579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa659c3-63f7-4168-a9d7-518314d697c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:57.604209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa659c3-63f7-4168-a9d7-518314d697c4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:57.604484 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa659c3-63f7-4168-a9d7-518314d697c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:57.604755 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa659c3-63f7-4168-a9d7-518314d697c4 None admin] PATH_INFO: `/v3/users/b4917bf683594d41b65e7ac042229ea6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:57.606007 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa659c3-63f7-4168-a9d7-518314d697c4 None admin] RBAC: Authorizing `identity:get_user(user_id=b4917bf683594d41b65e7ac042229ea6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:57.606577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f408d4-485f-4fcd-b698-4d0d01b32a74 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:57.607484 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa659c3-63f7-4168-a9d7-518314d697c4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:57.609894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 128/258] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:57 2022] GET /identity/v3/users/b4917bf683594d41b65e7ac042229ea6 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:57.610356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa659c3-63f7-4168-a9d7-518314d697c4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:57.642056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8b6db-8ab4-4ca2-a3a0-ff71eb828943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:57.643010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8b6db-8ab4-4ca2-a3a0-ff71eb828943 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:57.643464 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8b6db-8ab4-4ca2-a3a0-ff71eb828943 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:57.643907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8b6db-8ab4-4ca2-a3a0-ff71eb828943 None admin] PATH_INFO: `/v3/users/96a9af3f683345c393c32d2d15c05551` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:57.645282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8b6db-8ab4-4ca2-a3a0-ff71eb828943 None admin] RBAC: Authorizing `identity:get_user(user_id=96a9af3f683345c393c32d2d15c05551)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:57.646911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8b6db-8ab4-4ca2-a3a0-ff71eb828943 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:57.649001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 131/259] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:57 2022] GET /identity/v3/users/96a9af3f683345c393c32d2d15c05551 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:57.650052 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8b6db-8ab4-4ca2-a3a0-ff71eb828943 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:57.736442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa659c3-63f7-4168-a9d7-518314d697c4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:57.771735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ce897-61e6-44ec-8e49-0e24f60f99aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:57.772357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ce897-61e6-44ec-8e49-0e24f60f99aa None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:57.772632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ce897-61e6-44ec-8e49-0e24f60f99aa None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:57.772900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ce897-61e6-44ec-8e49-0e24f60f99aa None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:57.773600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ce897-61e6-44ec-8e49-0e24f60f99aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:57.773900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ce897-61e6-44ec-8e49-0e24f60f99aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:57.775391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ce897-61e6-44ec-8e49-0e24f60f99aa None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:57.784362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 129/260] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:14:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:57.785248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ce897-61e6-44ec-8e49-0e24f60f99aa None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:57.839465 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:57.883264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8b6db-8ab4-4ca2-a3a0-ff71eb828943 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:57.889623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ce897-61e6-44ec-8e49-0e24f60f99aa None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:57.935683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b5d0b-583e-4a68-91dd-f16cf1807ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:57.936783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b5d0b-583e-4a68-91dd-f16cf1807ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:57.937284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b5d0b-583e-4a68-91dd-f16cf1807ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:57.937907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b5d0b-583e-4a68-91dd-f16cf1807ad3 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:57.941959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b5d0b-583e-4a68-91dd-f16cf1807ad3 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:57.942861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f247a-8fcc-4bec-b8f4-bc04ccce5e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:57.943489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f247a-8fcc-4bec-b8f4-bc04ccce5e42 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:57.943769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f247a-8fcc-4bec-b8f4-bc04ccce5e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:57.944037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f247a-8fcc-4bec-b8f4-bc04ccce5e42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:57.944646 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b5d0b-583e-4a68-91dd-f16cf1807ad3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:57.946905 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 132/261] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:57 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:57.947892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b5d0b-583e-4a68-91dd-f16cf1807ad3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:57.984122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:58.085431 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f247a-8fcc-4bec-b8f4-bc04ccce5e42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:58.086970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f247a-8fcc-4bec-b8f4-bc04ccce5e42 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:58.132998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b5d0b-583e-4a68-91dd-f16cf1807ad3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:58.189512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:58.209531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56aedd-e378-4aac-8185-dc71009dae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:58.210504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56aedd-e378-4aac-8185-dc71009dae7c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:58.211021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56aedd-e378-4aac-8185-dc71009dae7c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:58.211506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56aedd-e378-4aac-8185-dc71009dae7c None admin] PATH_INFO: `/v3/users/96a9af3f683345c393c32d2d15c05551` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:58.217865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56aedd-e378-4aac-8185-dc71009dae7c None admin] RBAC: Authorizing `identity:get_user(user_id=96a9af3f683345c393c32d2d15c05551)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:58.219531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56aedd-e378-4aac-8185-dc71009dae7c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:58.221413 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 133/262] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:14:57 2022] GET /identity/v3/users/96a9af3f683345c393c32d2d15c05551 => generated 283 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:58.222351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56aedd-e378-4aac-8185-dc71009dae7c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:58.247277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 130/263] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:58.248976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f247a-8fcc-4bec-b8f4-bc04ccce5e42 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:58.397547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56aedd-e378-4aac-8185-dc71009dae7c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:58.421184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:58.423575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f247a-8fcc-4bec-b8f4-bc04ccce5e42 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:58.449120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa2003-2c4b-4933-a85d-0361812a1724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:58.449789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa2003-2c4b-4933-a85d-0361812a1724 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:58.450069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa2003-2c4b-4933-a85d-0361812a1724 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:58.450341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa2003-2c4b-4933-a85d-0361812a1724 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104/users/b4917bf683594d41b65e7ac042229ea6/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:58.454662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa2003-2c4b-4933-a85d-0361812a1724 None admin] RBAC: Authorizing `identity:create_grant(project_id=96874534e5534d3699fb67c58215c104, user_id=b4917bf683594d41b65e7ac042229ea6, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:58.456129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa2003-2c4b-4933-a85d-0361812a1724 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:58.479749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 134/264] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:14:58 2022] PUT /identity/v3/projects/96874534e5534d3699fb67c58215c104/users/b4917bf683594d41b65e7ac042229ea6/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:14:58.481654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa2003-2c4b-4933-a85d-0361812a1724 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:58.494344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769806c3-b7a3-4847-8a31-8640be545b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:58.494979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769806c3-b7a3-4847-8a31-8640be545b39 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:58.495256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769806c3-b7a3-4847-8a31-8640be545b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:58.495523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769806c3-b7a3-4847-8a31-8640be545b39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:58.521493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:58.571974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769806c3-b7a3-4847-8a31-8640be545b39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:58.573946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769806c3-b7a3-4847-8a31-8640be545b39 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:58.678513 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 131/265] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:14:58.690421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769806c3-b7a3-4847-8a31-8640be545b39 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:58.728032 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa2003-2c4b-4933-a85d-0361812a1724 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:58.802075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:58.813564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e130299-5a6a-40f9-9dcc-b2d776ac28fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:58.814282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e130299-5a6a-40f9-9dcc-b2d776ac28fd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:58.814565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e130299-5a6a-40f9-9dcc-b2d776ac28fd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:58.814839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e130299-5a6a-40f9-9dcc-b2d776ac28fd None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:58.816012 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e130299-5a6a-40f9-9dcc-b2d776ac28fd None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:58.817491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e130299-5a6a-40f9-9dcc-b2d776ac28fd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:58.819419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 135/266] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:58 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:58.825423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e130299-5a6a-40f9-9dcc-b2d776ac28fd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:58.857463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:58.884069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769806c3-b7a3-4847-8a31-8640be545b39 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:58.953321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949824e-bcf5-469d-9f9b-c22938d228c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:58.953880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949824e-bcf5-469d-9f9b-c22938d228c1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:58.954152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949824e-bcf5-469d-9f9b-c22938d228c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:58.954425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949824e-bcf5-469d-9f9b-c22938d228c1 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:58.956222 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949824e-bcf5-469d-9f9b-c22938d228c1 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:58.957711 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949824e-bcf5-469d-9f9b-c22938d228c1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:58.959343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 132/267] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:14:58 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:58.965341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949824e-bcf5-469d-9f9b-c22938d228c1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:58.994394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e130299-5a6a-40f9-9dcc-b2d776ac28fd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:59.060632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433df4f1-5a01-4b10-ad7e-4865dd507edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:59.061262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433df4f1-5a01-4b10-ad7e-4865dd507edb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:59.061534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433df4f1-5a01-4b10-ad7e-4865dd507edb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:59.061792 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433df4f1-5a01-4b10-ad7e-4865dd507edb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:59.062526 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433df4f1-5a01-4b10-ad7e-4865dd507edb None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=96874534e5534d3699fb67c58215c104, user.id=96a9af3f683345c393c32d2d15c05551) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:59.062832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433df4f1-5a01-4b10-ad7e-4865dd507edb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:59.064252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433df4f1-5a01-4b10-ad7e-4865dd507edb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:59.073933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 136/268] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:58 2022] GET /identity/v3/role_assignments?user.id=96a9af3f683345c393c32d2d15c05551&scope.project.id=96874534e5534d3699fb67c58215c104&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:14:59.131636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949824e-bcf5-469d-9f9b-c22938d228c1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:14:59.158103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:14:59.183147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4e316-a676-4518-9330-1a269ea442b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:14:59.183779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4e316-a676-4518-9330-1a269ea442b7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:59.184043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4e316-a676-4518-9330-1a269ea442b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:59.184358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4e316-a676-4518-9330-1a269ea442b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:59.185121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4e316-a676-4518-9330-1a269ea442b7 None admin] RBAC: Adding query filter params (role.id=84212031716c4cf8a5083824fe47dbb7, scope.project.id=96874534e5534d3699fb67c58215c104, user.id=96a9af3f683345c393c32d2d15c05551) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:14:59.185431 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4e316-a676-4518-9330-1a269ea442b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:14:59.186905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4e316-a676-4518-9330-1a269ea442b7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:14:59.197153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 133/269] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:14:58 2022] GET /identity/v3/role_assignments?user.id=96a9af3f683345c393c32d2d15c05551&scope.project.id=96874534e5534d3699fb67c58215c104&role.id=84212031716c4cf8a5083824fe47dbb7 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:14:59.549444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dc487-473d-4129-8270-8eb657e18cb4 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:59.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dc487-473d-4129-8270-8eb657e18cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:59.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dc487-473d-4129-8270-8eb657e18cb4 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:59.551338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 137/270] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:59.558563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b1459-9517-4768-86d5-8fbb2e845234 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:59.559082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b1459-9517-4768-86d5-8fbb2e845234 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:59.559082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b1459-9517-4768-86d5-8fbb2e845234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:59.581863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-509b1459-9517-4768-86d5-8fbb2e845234 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:59.583084 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-509b1459-9517-4768-86d5-8fbb2e845234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:59.666710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 134/271] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:59.776260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cc42d-bcca-4a6e-8944-161ce271ecce None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:59.776582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cc42d-bcca-4a6e-8944-161ce271ecce None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:59.776806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cc42d-bcca-4a6e-8944-161ce271ecce None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:59.778179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 138/272] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:14:59.785027 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabd56b-95b5-4edc-bb75-dd5f8c97cea2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:59.785596 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabd56b-95b5-4edc-bb75-dd5f8c97cea2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:59.786059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabd56b-95b5-4edc-bb75-dd5f8c97cea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:59.807880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1cabd56b-95b5-4edc-bb75-dd5f8c97cea2 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:59.809396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1cabd56b-95b5-4edc-bb75-dd5f8c97cea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:59.834456 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a168c45-bdac-45f4-bcec-82e2eea21beb None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:59.835089 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a168c45-bdac-45f4-bcec-82e2eea21beb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:59.835551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a168c45-bdac-45f4-bcec-82e2eea21beb None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:59.837126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 139/273] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:14:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:14:59.848024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6421fb4-f51b-4e3d-bdd3-717c00ef7ae3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:14:59.848637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6421fb4-f51b-4e3d-bdd3-717c00ef7ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:14:59.849184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6421fb4-f51b-4e3d-bdd3-717c00ef7ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:14:59.880027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d6421fb4-f51b-4e3d-bdd3-717c00ef7ae3 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:14:59.881675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d6421fb4-f51b-4e3d-bdd3-717c00ef7ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:14:59.891406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 135/274] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:14:59.969954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabd56b-95b5-4edc-bb75-dd5f8c97cea2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:14:59.977111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 140/275] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:00.024932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:00.078376 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabd56b-95b5-4edc-bb75-dd5f8c97cea2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:00.111265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685906fd-a4e1-4f17-8b89-a6b53a1077ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:00.111896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685906fd-a4e1-4f17-8b89-a6b53a1077ee None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:00.112166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685906fd-a4e1-4f17-8b89-a6b53a1077ee None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:00.112429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685906fd-a4e1-4f17-8b89-a6b53a1077ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:00.122900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685906fd-a4e1-4f17-8b89-a6b53a1077ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-685906fd-a4e1-4f17-8b89-a6b53a1077ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:00.127648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:00.128521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:00.129423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 136/276] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:14:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:00.134850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6421fb4-f51b-4e3d-bdd3-717c00ef7ae3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:00.147165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:00.172451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685906fd-a4e1-4f17-8b89-a6b53a1077ee None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:00.248162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6421fb4-f51b-4e3d-bdd3-717c00ef7ae3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:00.284415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600c6fb-6b66-4e06-b1c7-74e7ff0a4a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:00.285030 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600c6fb-6b66-4e06-b1c7-74e7ff0a4a1f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:00.285331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600c6fb-6b66-4e06-b1c7-74e7ff0a4a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:00.285594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600c6fb-6b66-4e06-b1c7-74e7ff0a4a1f None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:00.286255 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600c6fb-6b66-4e06-b1c7-74e7ff0a4a1f None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:00.286642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600c6fb-6b66-4e06-b1c7-74e7ff0a4a1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:00.288125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600c6fb-6b66-4e06-b1c7-74e7ff0a4a1f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:00.292962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685906fd-a4e1-4f17-8b89-a6b53a1077ee None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:00.295307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:00.302385 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 141/277] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:00.302385 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600c6fb-6b66-4e06-b1c7-74e7ff0a4a1f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:00.330841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4c064-356c-4a56-9046-e6e386e28837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:00.331396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4c064-356c-4a56-9046-e6e386e28837 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:00.331668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4c064-356c-4a56-9046-e6e386e28837 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:00.331935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4c064-356c-4a56-9046-e6e386e28837 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:00.333277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a4c064-356c-4a56-9046-e6e386e28837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:00.334719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a4c064-356c-4a56-9046-e6e386e28837 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:00.336546 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 137/278] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:00.341751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4c064-356c-4a56-9046-e6e386e28837 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:00.372125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:00.401488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600c6fb-6b66-4e06-b1c7-74e7ff0a4a1f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:00.432957 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eed61-4383-472a-8444-3a5127503539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:00.433611 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eed61-4383-472a-8444-3a5127503539 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:00.433885 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eed61-4383-472a-8444-3a5127503539 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:00.434153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eed61-4383-472a-8444-3a5127503539 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:00.444092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48eed61-4383-472a-8444-3a5127503539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:00.445045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4c064-356c-4a56-9046-e6e386e28837 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c48eed61-4383-472a-8444-3a5127503539 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:00.449390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:00.450251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:00.451031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 142/279] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:00.451691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eed61-4383-472a-8444-3a5127503539 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:00.486047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741c6b4-7107-4177-8bd7-2e34fe4a436c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:00.486685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741c6b4-7107-4177-8bd7-2e34fe4a436c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:00.486958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741c6b4-7107-4177-8bd7-2e34fe4a436c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:00.487243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741c6b4-7107-4177-8bd7-2e34fe4a436c None admin] PATH_INFO: `/v3/users/3bd43df9126c451ea39e84b6fd08e9f4` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:00.502128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741c6b4-7107-4177-8bd7-2e34fe4a436c None admin] RBAC: Authorizing `identity:get_user(user_id=3bd43df9126c451ea39e84b6fd08e9f4)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:00.503577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741c6b4-7107-4177-8bd7-2e34fe4a436c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:00.510135 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 138/280] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:15:00 2022] GET /identity/v3/users/3bd43df9126c451ea39e84b6fd08e9f4 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:00.510135 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741c6b4-7107-4177-8bd7-2e34fe4a436c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:00.539896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:00.557353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eed61-4383-472a-8444-3a5127503539 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:00.587462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:00.593499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b47ae6-96b5-45c0-a2c1-67cc51eceeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:00.594098 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b47ae6-96b5-45c0-a2c1-67cc51eceeaf None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:00.594382 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b47ae6-96b5-45c0-a2c1-67cc51eceeaf None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:00.594654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b47ae6-96b5-45c0-a2c1-67cc51eceeaf None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:00.595445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b47ae6-96b5-45c0-a2c1-67cc51eceeaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:00.596773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b47ae6-96b5-45c0-a2c1-67cc51eceeaf None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:00.616969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741c6b4-7107-4177-8bd7-2e34fe4a436c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:00.629130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 143/281] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:15:00 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:15:00.629130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b47ae6-96b5-45c0-a2c1-67cc51eceeaf None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:00.660228 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afcebc-f1ef-4260-82c2-0d2be2d1b202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:00.660963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afcebc-f1ef-4260-82c2-0d2be2d1b202 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:00.661331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afcebc-f1ef-4260-82c2-0d2be2d1b202 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:00.661668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afcebc-f1ef-4260-82c2-0d2be2d1b202 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:00.662434 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afcebc-f1ef-4260-82c2-0d2be2d1b202 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:00.663225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afcebc-f1ef-4260-82c2-0d2be2d1b202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:00.664479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afcebc-f1ef-4260-82c2-0d2be2d1b202 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:00.684738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 139/282] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:00.686132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afcebc-f1ef-4260-82c2-0d2be2d1b202 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:00.773669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b47ae6-96b5-45c0-a2c1-67cc51eceeaf None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:00.794899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:00.806352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afcebc-f1ef-4260-82c2-0d2be2d1b202 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:00.823647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b123bd-594f-4cdc-8e74-026b10e1cf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:00.824754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b123bd-594f-4cdc-8e74-026b10e1cf9c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:00.824754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b123bd-594f-4cdc-8e74-026b10e1cf9c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:00.824946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b123bd-594f-4cdc-8e74-026b10e1cf9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:00.826229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b123bd-594f-4cdc-8e74-026b10e1cf9c None admin] RBAC: Adding query filter params (role.id=231a681c35614ea3aaf3452a8c167c5c, user.id=3bd43df9126c451ea39e84b6fd08e9f4, scope.system=all) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:00.826229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b123bd-594f-4cdc-8e74-026b10e1cf9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:00.827356 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b123bd-594f-4cdc-8e74-026b10e1cf9c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:00.833752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 144/283] 10.222.0.60 () {56 vars in 1308 bytes} [Sat Sep 17 04:15:00 2022] GET /identity/v3/role_assignments?user.id=3bd43df9126c451ea39e84b6fd08e9f4&scope.system=all&role.id=231a681c35614ea3aaf3452a8c167c5c => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:00.855915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283025a4-1d5a-4a1e-9c71-bcb6467eeccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:00.856494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283025a4-1d5a-4a1e-9c71-bcb6467eeccd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:00.856770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283025a4-1d5a-4a1e-9c71-bcb6467eeccd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:00.857038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283025a4-1d5a-4a1e-9c71-bcb6467eeccd None admin] PATH_INFO: `/v3/users/d69c2706136b483cb68f173575f16333` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:00.858273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283025a4-1d5a-4a1e-9c71-bcb6467eeccd None admin] RBAC: Authorizing `identity:get_user(user_id=d69c2706136b483cb68f173575f16333)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:00.859759 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283025a4-1d5a-4a1e-9c71-bcb6467eeccd None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:00.861698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 140/284] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:15:00 2022] GET /identity/v3/users/d69c2706136b483cb68f173575f16333 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:00.868090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b123bd-594f-4cdc-8e74-026b10e1cf9c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:00.928477 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:00.982156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b123bd-594f-4cdc-8e74-026b10e1cf9c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:01.021542 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022110c3-e5d3-49f6-9a14-934f26f2e229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:01.022189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022110c3-e5d3-49f6-9a14-934f26f2e229 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:01.022470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022110c3-e5d3-49f6-9a14-934f26f2e229 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:01.022739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022110c3-e5d3-49f6-9a14-934f26f2e229 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:01.023976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022110c3-e5d3-49f6-9a14-934f26f2e229 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:01.025528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022110c3-e5d3-49f6-9a14-934f26f2e229 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:01.029179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 145/285] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:00 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:01.033440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283025a4-1d5a-4a1e-9c71-bcb6467eeccd None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:01.160219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283025a4-1d5a-4a1e-9c71-bcb6467eeccd None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:01.194259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c66955-c6dd-46c4-831d-30119f9072a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:01.194931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c66955-c6dd-46c4-831d-30119f9072a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:01.195207 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c66955-c6dd-46c4-831d-30119f9072a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:01.195477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c66955-c6dd-46c4-831d-30119f9072a2 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104/users/d69c2706136b483cb68f173575f16333/roles/7dae07bb44b2421baecc0522aaf94e7b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:01.197480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c66955-c6dd-46c4-831d-30119f9072a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=96874534e5534d3699fb67c58215c104, user_id=d69c2706136b483cb68f173575f16333, role_id=7dae07bb44b2421baecc0522aaf94e7b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:01.198924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c66955-c6dd-46c4-831d-30119f9072a2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:01.217202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c2d68-18da-46da-a5ee-d1672730d128 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:01.217202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c2d68-18da-46da-a5ee-d1672730d128 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:01.219053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c2d68-18da-46da-a5ee-d1672730d128 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:01.219053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 146/286] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:01.224674 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1e251-3945-4a8c-bbb9-3d107d4ca24b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:01.225275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1e251-3945-4a8c-bbb9-3d107d4ca24b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:01.225738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1e251-3945-4a8c-bbb9-3d107d4ca24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:01.235014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 141/287] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:15:01 2022] PUT /identity/v3/projects/96874534e5534d3699fb67c58215c104/users/d69c2706136b483cb68f173575f16333/roles/7dae07bb44b2421baecc0522aaf94e7b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:15:01.246966 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-00c1e251-3945-4a8c-bbb9-3d107d4ca24b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:01.248532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-00c1e251-3945-4a8c-bbb9-3d107d4ca24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:01.344119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 147/288] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:01.639152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c66955-c6dd-46c4-831d-30119f9072a2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:01.680449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:01.784249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c66955-c6dd-46c4-831d-30119f9072a2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:01.821783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f9503-2ec9-4c37-94df-85938e5d80c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:01.822424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f9503-2ec9-4c37-94df-85938e5d80c5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:01.822710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f9503-2ec9-4c37-94df-85938e5d80c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:01.822979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f9503-2ec9-4c37-94df-85938e5d80c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:01.833619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f9503-2ec9-4c37-94df-85938e5d80c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-567f9503-2ec9-4c37-94df-85938e5d80c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:01.839289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:01.840283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:01.841946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 142/289] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:01.846444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1e251-3945-4a8c-bbb9-3d107d4ca24b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:01.898031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:01.958297 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1e251-3945-4a8c-bbb9-3d107d4ca24b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:02.001956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd3b22-c960-475e-a25d-d1312258bf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:02.002584 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd3b22-c960-475e-a25d-d1312258bf9d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:02.002859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd3b22-c960-475e-a25d-d1312258bf9d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:02.003129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd3b22-c960-475e-a25d-d1312258bf9d None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:02.003866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd3b22-c960-475e-a25d-d1312258bf9d None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:02.004165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd3b22-c960-475e-a25d-d1312258bf9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:02.005632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd3b22-c960-475e-a25d-d1312258bf9d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:02.014881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 148/290] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:02.022211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f9503-2ec9-4c37-94df-85938e5d80c5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:02.126726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:02.151871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f9503-2ec9-4c37-94df-85938e5d80c5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:02.191374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c41a50-b59f-4997-9c70-10ec46af253b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:02.192009 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c41a50-b59f-4997-9c70-10ec46af253b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:02.192286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c41a50-b59f-4997-9c70-10ec46af253b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:02.192565 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c41a50-b59f-4997-9c70-10ec46af253b None admin] PATH_INFO: `/v3/users/b4917bf683594d41b65e7ac042229ea6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:02.193693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c41a50-b59f-4997-9c70-10ec46af253b None admin] RBAC: Authorizing `identity:get_user(user_id=b4917bf683594d41b65e7ac042229ea6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:02.195136 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c41a50-b59f-4997-9c70-10ec46af253b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:02.197075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 143/291] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:15:02 2022] GET /identity/v3/users/b4917bf683594d41b65e7ac042229ea6 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:02.202687 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd3b22-c960-475e-a25d-d1312258bf9d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:02.316442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd3b22-c960-475e-a25d-d1312258bf9d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:02.333761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:02.365178 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f748192-78c9-49d1-a815-2c863380b27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:02.365795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f748192-78c9-49d1-a815-2c863380b27b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:02.366069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f748192-78c9-49d1-a815-2c863380b27b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:02.366338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f748192-78c9-49d1-a815-2c863380b27b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:02.442416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f748192-78c9-49d1-a815-2c863380b27b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:02.443796 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f748192-78c9-49d1-a815-2c863380b27b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:02.539469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 149/292] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:02.546225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c41a50-b59f-4997-9c70-10ec46af253b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:02.729857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c41a50-b59f-4997-9c70-10ec46af253b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:02.773855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f230f4-6a7c-4428-82e1-edce5aa46f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:02.774498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f230f4-6a7c-4428-82e1-edce5aa46f3f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:02.774801 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f230f4-6a7c-4428-82e1-edce5aa46f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:02.775073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f230f4-6a7c-4428-82e1-edce5aa46f3f None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:02.776210 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f230f4-6a7c-4428-82e1-edce5aa46f3f None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:02.777789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f230f4-6a7c-4428-82e1-edce5aa46f3f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:02.779640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 144/293] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:02 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:02.785358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f748192-78c9-49d1-a815-2c863380b27b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:02.809685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:02.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f748192-78c9-49d1-a815-2c863380b27b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:02.948609 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bc9aa-74df-43c1-b2e1-dee36c3accbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:02.949248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bc9aa-74df-43c1-b2e1-dee36c3accbf None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:02.949521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bc9aa-74df-43c1-b2e1-dee36c3accbf None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:02.949794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bc9aa-74df-43c1-b2e1-dee36c3accbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:02.950508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bc9aa-74df-43c1-b2e1-dee36c3accbf None admin] RBAC: Adding query filter params (role.id=231a681c35614ea3aaf3452a8c167c5c, scope.project.id=96874534e5534d3699fb67c58215c104, user.id=b4917bf683594d41b65e7ac042229ea6) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:02.950811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bc9aa-74df-43c1-b2e1-dee36c3accbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:02.952249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bc9aa-74df-43c1-b2e1-dee36c3accbf None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:02.961682 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 150/294] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:02 2022] GET /identity/v3/role_assignments?user.id=b4917bf683594d41b65e7ac042229ea6&scope.project.id=96874534e5534d3699fb67c58215c104&role.id=231a681c35614ea3aaf3452a8c167c5c => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:03.324946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10976f7-8272-420c-9246-6bbe6658cc82 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:03.325364 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10976f7-8272-420c-9246-6bbe6658cc82 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:03.325554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10976f7-8272-420c-9246-6bbe6658cc82 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:03.327147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 145/295] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:03.332698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efcf62-1d5b-4f44-baa0-06f308c144f1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:03.332982 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efcf62-1d5b-4f44-baa0-06f308c144f1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:03.333280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efcf62-1d5b-4f44-baa0-06f308c144f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:03.354331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b5efcf62-1d5b-4f44-baa0-06f308c144f1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:03.355519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b5efcf62-1d5b-4f44-baa0-06f308c144f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:03.424681 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 151/296] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:03.609121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d09a97-3520-4c48-97ae-2d53c75012f3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:03.609121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d09a97-3520-4c48-97ae-2d53c75012f3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:03.609121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d09a97-3520-4c48-97ae-2d53c75012f3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:03.609884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 146/297] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:03.614340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5927a92-8aea-4d90-9708-5084c820c0d7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:03.614686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5927a92-8aea-4d90-9708-5084c820c0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:03.614963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5927a92-8aea-4d90-9708-5084c820c0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:03.630626 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:03.637005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e5927a92-8aea-4d90-9708-5084c820c0d7 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:03.640628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e5927a92-8aea-4d90-9708-5084c820c0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:03.700732 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d09a97-3520-4c48-97ae-2d53c75012f3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:03.723825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:03.726239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 152/298] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:03.807277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c35a9-952d-41dc-ae1f-dbeb1bb8cacb None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:03.807655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c35a9-952d-41dc-ae1f-dbeb1bb8cacb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:03.807985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c35a9-952d-41dc-ae1f-dbeb1bb8cacb None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:03.809569 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d09a97-3520-4c48-97ae-2d53c75012f3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:03.810130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 153/299] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:15:03.814785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeffbcd-a91e-4d8e-8174-5a44727b1bcb None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:03.815065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeffbcd-a91e-4d8e-8174-5a44727b1bcb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:03.815338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeffbcd-a91e-4d8e-8174-5a44727b1bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:03.836135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-3eeffbcd-a91e-4d8e-8174-5a44727b1bcb None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:03.837577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3eeffbcd-a91e-4d8e-8174-5a44727b1bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:03.845671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb16ce6-ead0-41e2-bddb-a198a471ee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:03.846563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb16ce6-ead0-41e2-bddb-a198a471ee5a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:03.846904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb16ce6-ead0-41e2-bddb-a198a471ee5a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:03.847237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb16ce6-ead0-41e2-bddb-a198a471ee5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:03.858237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb16ce6-ead0-41e2-bddb-a198a471ee5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-9bb16ce6-ead0-41e2-bddb-a198a471ee5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:03.863659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:03.864567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:03.865382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 147/300] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:03.872307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb16ce6-ead0-41e2-bddb-a198a471ee5a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:03.937196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 154/301] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:03.965453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:03.984784 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb16ce6-ead0-41e2-bddb-a198a471ee5a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:04.010080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e11a2b-f5b2-4353-9ff6-fd134a31d24a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:04.010675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e11a2b-f5b2-4353-9ff6-fd134a31d24a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:04.011167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e11a2b-f5b2-4353-9ff6-fd134a31d24a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:04.012987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 155/302] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:04.013865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e11a2b-f5b2-4353-9ff6-fd134a31d24a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:04.023570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48614c84-6f28-4659-9470-65e3dd2dbc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:04.024478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48614c84-6f28-4659-9470-65e3dd2dbc68 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:04.025840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48614c84-6f28-4659-9470-65e3dd2dbc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:04.026316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48614c84-6f28-4659-9470-65e3dd2dbc68 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:04.027630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48614c84-6f28-4659-9470-65e3dd2dbc68 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:04.028374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48614c84-6f28-4659-9470-65e3dd2dbc68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:04.030284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48614c84-6f28-4659-9470-65e3dd2dbc68 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:04.042836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 148/303] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:04.044730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db50d2-ee97-47ea-b9a9-0bbb3cc420a6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:04.045192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db50d2-ee97-47ea-b9a9-0bbb3cc420a6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:04.045617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db50d2-ee97-47ea-b9a9-0bbb3cc420a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:04.064168 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-01db50d2-ee97-47ea-b9a9-0bbb3cc420a6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:04.065492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-01db50d2-ee97-47ea-b9a9-0bbb3cc420a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:04.091884 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:04.128392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e11a2b-f5b2-4353-9ff6-fd134a31d24a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:04.138074 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 149/304] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:04.150308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db50d2-ee97-47ea-b9a9-0bbb3cc420a6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:04.162561 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a114cdb-9b89-4c58-84ed-48b1e8f8abe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:04.163252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a114cdb-9b89-4c58-84ed-48b1e8f8abe0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:04.163527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a114cdb-9b89-4c58-84ed-48b1e8f8abe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:04.163793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a114cdb-9b89-4c58-84ed-48b1e8f8abe0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:04.164981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a114cdb-9b89-4c58-84ed-48b1e8f8abe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:04.166624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a114cdb-9b89-4c58-84ed-48b1e8f8abe0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:04.168367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 156/305] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:04.173747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a114cdb-9b89-4c58-84ed-48b1e8f8abe0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:04.231661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:04.265777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db50d2-ee97-47ea-b9a9-0bbb3cc420a6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:04.275153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a114cdb-9b89-4c58-84ed-48b1e8f8abe0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:04.301625 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd7a51-8b3e-406c-9276-9bf32bc66d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:04.302260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd7a51-8b3e-406c-9276-9bf32bc66d1a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:04.302536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd7a51-8b3e-406c-9276-9bf32bc66d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:04.302848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd7a51-8b3e-406c-9276-9bf32bc66d1a None admin] PATH_INFO: `/v3/users/3bd43df9126c451ea39e84b6fd08e9f4` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:04.303964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd7a51-8b3e-406c-9276-9bf32bc66d1a None admin] RBAC: Authorizing `identity:get_user(user_id=3bd43df9126c451ea39e84b6fd08e9f4)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:04.305479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd7a51-8b3e-406c-9276-9bf32bc66d1a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:04.307094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 150/306] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:15:04 2022] GET /identity/v3/users/3bd43df9126c451ea39e84b6fd08e9f4 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:04.307848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd7a51-8b3e-406c-9276-9bf32bc66d1a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:04.316962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab8bbf-9bee-40b0-8bc1-25b89f48d31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:04.317827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab8bbf-9bee-40b0-8bc1-25b89f48d31e None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:04.318622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab8bbf-9bee-40b0-8bc1-25b89f48d31e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:04.319077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab8bbf-9bee-40b0-8bc1-25b89f48d31e None admin] PATH_INFO: `/v3/groups` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:04.320409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab8bbf-9bee-40b0-8bc1-25b89f48d31e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:04.322206 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab8bbf-9bee-40b0-8bc1-25b89f48d31e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:04.326745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:04.342007 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 157/307] 10.222.0.60 () {60 vars in 1134 bytes} [Sat Sep 17 04:15:04 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:15:04.354188 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab8bbf-9bee-40b0-8bc1-25b89f48d31e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:04.426723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd7a51-8b3e-406c-9276-9bf32bc66d1a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:04.465320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2155e-74cd-4351-a887-94af24c01681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:04.465980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2155e-74cd-4351-a887-94af24c01681 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:04.466255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2155e-74cd-4351-a887-94af24c01681 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:04.466676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2155e-74cd-4351-a887-94af24c01681 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:04.470763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:04.477366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab8bbf-9bee-40b0-8bc1-25b89f48d31e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:04.483137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2155e-74cd-4351-a887-94af24c01681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-67c2155e-74cd-4351-a887-94af24c01681 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:04.490170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:04.491144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:04.491925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:04.491925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:04.491925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:04.491925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:04.491925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:04.491925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:04.491925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:04.491925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:04.491925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:04.491925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:04.491925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:15:04.491925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:04.493605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 151/308] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:04.494047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2155e-74cd-4351-a887-94af24c01681 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:04.522001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37825eed-ec62-4749-bbd0-57bb4230924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:04.522802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37825eed-ec62-4749-bbd0-57bb4230924f None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:04.523077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37825eed-ec62-4749-bbd0-57bb4230924f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:04.523349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37825eed-ec62-4749-bbd0-57bb4230924f None admin] PATH_INFO: `/v3/system/users/3bd43df9126c451ea39e84b6fd08e9f4/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:04.524794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37825eed-ec62-4749-bbd0-57bb4230924f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3bd43df9126c451ea39e84b6fd08e9f4, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:04.526207 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37825eed-ec62-4749-bbd0-57bb4230924f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:04.536739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 158/309] 10.222.0.60 () {58 vars in 1334 bytes} [Sat Sep 17 04:15:04 2022] PUT /identity/v3/system/users/3bd43df9126c451ea39e84b6fd08e9f4/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:15:04.537408 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37825eed-ec62-4749-bbd0-57bb4230924f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:04.617422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2155e-74cd-4351-a887-94af24c01681 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:04.618170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:04.663838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c5fe7-10f5-4b3c-bbe6-42ff8ed753f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:04.666987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c5fe7-10f5-4b3c-bbe6-42ff8ed753f0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:04.666987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c5fe7-10f5-4b3c-bbe6-42ff8ed753f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:04.666987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c5fe7-10f5-4b3c-bbe6-42ff8ed753f0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:04.677208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37825eed-ec62-4749-bbd0-57bb4230924f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:04.682254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c5fe7-10f5-4b3c-bbe6-42ff8ed753f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c67c5fe7-10f5-4b3c-bbe6-42ff8ed753f0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:04.689730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:04.690675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:04.691466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:04.691466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:04.691466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:04.691466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:04.691466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:04.691466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:04.691466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:04.691466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:04.691466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:04.691466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:04.691466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:15:04.691466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:04.692065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 152/310] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:04.697634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c5fe7-10f5-4b3c-bbe6-42ff8ed753f0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:04.717581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c28199-f050-437d-be3b-292133239b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:04.718235 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c28199-f050-437d-be3b-292133239b0a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:04.718516 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c28199-f050-437d-be3b-292133239b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:04.718793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c28199-f050-437d-be3b-292133239b0a None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:04.719478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c28199-f050-437d-be3b-292133239b0a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:04.719794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c28199-f050-437d-be3b-292133239b0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:04.721216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c28199-f050-437d-be3b-292133239b0a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:04.733179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 159/311] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:04.736502 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c28199-f050-437d-be3b-292133239b0a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:04.766991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:04.821856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c5fe7-10f5-4b3c-bbe6-42ff8ed753f0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:04.865739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c28199-f050-437d-be3b-292133239b0a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:04.869963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b5287-5128-4e58-9490-b6b36bbdc28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:04.871014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b5287-5128-4e58-9490-b6b36bbdc28b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:04.871014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b5287-5128-4e58-9490-b6b36bbdc28b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:04.871208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b5287-5128-4e58-9490-b6b36bbdc28b None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:04.871876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b5287-5128-4e58-9490-b6b36bbdc28b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:04.872176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b5287-5128-4e58-9490-b6b36bbdc28b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:04.873607 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b5287-5128-4e58-9490-b6b36bbdc28b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:04.883246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 153/312] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:04.888442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b5287-5128-4e58-9490-b6b36bbdc28b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:04.893482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:04.909679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb2cfc-66c1-45d5-87c8-76b600178a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:04.910618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb2cfc-66c1-45d5-87c8-76b600178a40 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:04.911121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb2cfc-66c1-45d5-87c8-76b600178a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:04.911564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb2cfc-66c1-45d5-87c8-76b600178a40 None admin] PATH_INFO: `/v3/users/30153216ec294123b388e36f177b942b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:04.926551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb2cfc-66c1-45d5-87c8-76b600178a40 None admin] RBAC: Authorizing `identity:get_user(user_id=30153216ec294123b388e36f177b942b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:04.928422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb2cfc-66c1-45d5-87c8-76b600178a40 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:04.930441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 160/313] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:15:04 2022] GET /identity/v3/users/30153216ec294123b388e36f177b942b => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:04.938872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb2cfc-66c1-45d5-87c8-76b600178a40 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:04.996547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b5287-5128-4e58-9490-b6b36bbdc28b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:05.013988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:05.031809 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d7e80-c744-4ef1-8bee-33861fc839d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:05.032454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d7e80-c744-4ef1-8bee-33861fc839d5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:05.032733 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d7e80-c744-4ef1-8bee-33861fc839d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:05.033008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d7e80-c744-4ef1-8bee-33861fc839d5 None admin] PATH_INFO: `/v3/users/d69c2706136b483cb68f173575f16333` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:05.034225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d7e80-c744-4ef1-8bee-33861fc839d5 None admin] RBAC: Authorizing `identity:get_user(user_id=d69c2706136b483cb68f173575f16333)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:05.035687 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d7e80-c744-4ef1-8bee-33861fc839d5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:05.037385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 154/314] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:15:04 2022] GET /identity/v3/users/d69c2706136b483cb68f173575f16333 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:05.042841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d7e80-c744-4ef1-8bee-33861fc839d5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:05.050337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb2cfc-66c1-45d5-87c8-76b600178a40 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:05.087690 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42204b5d-d475-43ca-aee1-c694d3bef4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:05.088750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42204b5d-d475-43ca-aee1-c694d3bef4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:05.089285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42204b5d-d475-43ca-aee1-c694d3bef4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:05.089742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42204b5d-d475-43ca-aee1-c694d3bef4c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:05.090826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42204b5d-d475-43ca-aee1-c694d3bef4c1 None admin] RBAC: Adding query filter params (role.id=7dae07bb44b2421baecc0522aaf94e7b, user.id=30153216ec294123b388e36f177b942b, scope.system=all) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:05.091349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42204b5d-d475-43ca-aee1-c694d3bef4c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:05.093231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42204b5d-d475-43ca-aee1-c694d3bef4c1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:05.106225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 161/315] 10.222.0.60 () {56 vars in 1308 bytes} [Sat Sep 17 04:15:04 2022] GET /identity/v3/role_assignments?user.id=30153216ec294123b388e36f177b942b&scope.system=all&role.id=7dae07bb44b2421baecc0522aaf94e7b => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:05.167263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d7e80-c744-4ef1-8bee-33861fc839d5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:05.204289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4320e07-c100-432b-a58e-5439863f1c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:05.204835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4320e07-c100-432b-a58e-5439863f1c54 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:05.205137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4320e07-c100-432b-a58e-5439863f1c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:05.205420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4320e07-c100-432b-a58e-5439863f1c54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:05.265259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:05.279391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4320e07-c100-432b-a58e-5439863f1c54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:05.281017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4320e07-c100-432b-a58e-5439863f1c54 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:05.390718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 155/316] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 04:15:05.396499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42204b5d-d475-43ca-aee1-c694d3bef4c1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:05.439403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:05.495020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42204b5d-d475-43ca-aee1-c694d3bef4c1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:05.531439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60a21a-27e8-4bc3-ba8a-22d2794e2b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:05.532151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60a21a-27e8-4bc3-ba8a-22d2794e2b11 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:05.532500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60a21a-27e8-4bc3-ba8a-22d2794e2b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:05.532832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60a21a-27e8-4bc3-ba8a-22d2794e2b11 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:05.534070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60a21a-27e8-4bc3-ba8a-22d2794e2b11 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:05.535578 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60a21a-27e8-4bc3-ba8a-22d2794e2b11 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:05.537342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 162/317] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:05 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:05.543516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4320e07-c100-432b-a58e-5439863f1c54 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:05.634937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:05.652389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4320e07-c100-432b-a58e-5439863f1c54 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:05.690678 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6288a-134c-4793-bdac-ea80c3250449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:05.691563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6288a-134c-4793-bdac-ea80c3250449 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:05.691981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6288a-134c-4793-bdac-ea80c3250449 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:05.692379 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6288a-134c-4793-bdac-ea80c3250449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:05.693271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6288a-134c-4793-bdac-ea80c3250449 None admin] RBAC: Adding query filter params (role.id=7dae07bb44b2421baecc0522aaf94e7b, scope.project.id=96874534e5534d3699fb67c58215c104, user.id=d69c2706136b483cb68f173575f16333) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:05.693713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6288a-134c-4793-bdac-ea80c3250449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:05.695399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6288a-134c-4793-bdac-ea80c3250449 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:05.702159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 156/318] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:05 2022] GET /identity/v3/role_assignments?user.id=d69c2706136b483cb68f173575f16333&scope.project.id=96874534e5534d3699fb67c58215c104&role.id=7dae07bb44b2421baecc0522aaf94e7b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:06.593887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fbed5-0deb-4d70-a987-b592cb330060 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:06.594424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fbed5-0deb-4d70-a987-b592cb330060 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:06.594424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fbed5-0deb-4d70-a987-b592cb330060 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:06.595938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 163/319] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:06.601414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851214d1-0b88-44e6-91d6-de5590696f81 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:06.601821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851214d1-0b88-44e6-91d6-de5590696f81 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:06.602001 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851214d1-0b88-44e6-91d6-de5590696f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:06.621219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-851214d1-0b88-44e6-91d6-de5590696f81 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:06.622395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-851214d1-0b88-44e6-91d6-de5590696f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:06.702757 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 157/320] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:06.784813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cf683-317c-4dba-be39-718ed5f6a986 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:06.785187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cf683-317c-4dba-be39-718ed5f6a986 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:06.785368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cf683-317c-4dba-be39-718ed5f6a986 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:06.786942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 164/321] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:06.792102 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459666f-ed4c-4d31-8765-0a64a7d2d915 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:06.792384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459666f-ed4c-4d31-8765-0a64a7d2d915 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:06.792648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459666f-ed4c-4d31-8765-0a64a7d2d915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:06.811550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e459666f-ed4c-4d31-8765-0a64a7d2d915 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:06.812719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e459666f-ed4c-4d31-8765-0a64a7d2d915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:06.882233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 158/322] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:06.969887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cf683-317c-4dba-be39-718ed5f6a986 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:07.075868 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cf683-317c-4dba-be39-718ed5f6a986 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:07.086742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:07.111139 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944c323-4b93-4fb7-b82c-8986f4f954cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:07.111771 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944c323-4b93-4fb7-b82c-8986f4f954cb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:07.112046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944c323-4b93-4fb7-b82c-8986f4f954cb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:07.112308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944c323-4b93-4fb7-b82c-8986f4f954cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:07.113445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e944c323-4b93-4fb7-b82c-8986f4f954cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:07.114930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e944c323-4b93-4fb7-b82c-8986f4f954cb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:07.116667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 165/323] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:07.122533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459666f-ed4c-4d31-8765-0a64a7d2d915 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:07.135181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:07.141879 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944c323-4b93-4fb7-b82c-8986f4f954cb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:07.228277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459666f-ed4c-4d31-8765-0a64a7d2d915 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:07.245576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944c323-4b93-4fb7-b82c-8986f4f954cb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:07.266568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd6471-8372-4fa1-8784-2e144abab2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:07.267190 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd6471-8372-4fa1-8784-2e144abab2f9 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:07.267464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd6471-8372-4fa1-8784-2e144abab2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:07.267730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd6471-8372-4fa1-8784-2e144abab2f9 None admin] PATH_INFO: `/v3/groups` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:07.268385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd6471-8372-4fa1-8784-2e144abab2f9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:07.269873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd6471-8372-4fa1-8784-2e144abab2f9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:07.284171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61196f75-2274-4ad7-901e-26e7c9c453b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:07.284819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61196f75-2274-4ad7-901e-26e7c9c453b2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:07.285116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61196f75-2274-4ad7-901e-26e7c9c453b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:07.285391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61196f75-2274-4ad7-901e-26e7c9c453b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:07.289637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 159/324] 10.222.0.60 () {60 vars in 1134 bytes} [Sat Sep 17 04:15:07 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:15:07.296394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61196f75-2274-4ad7-901e-26e7c9c453b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-61196f75-2274-4ad7-901e-26e7c9c453b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:07.301523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:07.302531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:07.306671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 166/325] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:07.309725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd6471-8372-4fa1-8784-2e144abab2f9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:07.366219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c07f5a-b4ba-4217-b327-9c38067199a8 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:07.366557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c07f5a-b4ba-4217-b327-9c38067199a8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:07.367636 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c07f5a-b4ba-4217-b327-9c38067199a8 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:07.369300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 167/326] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:07.375030 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9425ef-358c-4320-b565-86bd7efb8404 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:07.375317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9425ef-358c-4320-b565-86bd7efb8404 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:07.375565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9425ef-358c-4320-b565-86bd7efb8404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:07.389157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:07.398008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-bc9425ef-358c-4320-b565-86bd7efb8404 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:07.399315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bc9425ef-358c-4320-b565-86bd7efb8404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:07.425227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd6471-8372-4fa1-8784-2e144abab2f9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:07.433481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:07.461333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7a22e-fff6-4087-883b-fac9534c9eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:07.461959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7a22e-fff6-4087-883b-fac9534c9eff None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:07.462266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7a22e-fff6-4087-883b-fac9534c9eff None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:07.462537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7a22e-fff6-4087-883b-fac9534c9eff None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:07.463211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7a22e-fff6-4087-883b-fac9534c9eff None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:07.463509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7a22e-fff6-4087-883b-fac9534c9eff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:07.464906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7a22e-fff6-4087-883b-fac9534c9eff None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:07.473637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 160/327] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:07.478452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 168/328] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:07.482727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7a22e-fff6-4087-883b-fac9534c9eff None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:07.587219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7a22e-fff6-4087-883b-fac9534c9eff None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:07.618906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:07.624458 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6f9d4-4e52-4eac-9daa-5149924374ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:07.625199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6f9d4-4e52-4eac-9daa-5149924374ca None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:07.625478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6f9d4-4e52-4eac-9daa-5149924374ca None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:07.625749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6f9d4-4e52-4eac-9daa-5149924374ca None admin] PATH_INFO: `/v3/users/3bd43df9126c451ea39e84b6fd08e9f4` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:07.626888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6f9d4-4e52-4eac-9daa-5149924374ca None admin] RBAC: Authorizing `identity:get_user(user_id=3bd43df9126c451ea39e84b6fd08e9f4)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:07.628360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6f9d4-4e52-4eac-9daa-5149924374ca None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:07.634875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 161/329] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:15:07 2022] GET /identity/v3/users/3bd43df9126c451ea39e84b6fd08e9f4 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:07.636279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9425ef-358c-4320-b565-86bd7efb8404 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:07.738029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9425ef-358c-4320-b565-86bd7efb8404 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:07.757030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6f9d4-4e52-4eac-9daa-5149924374ca None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:07.775333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f278df9-52be-4b9d-91d9-53dc48586305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:07.776066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f278df9-52be-4b9d-91d9-53dc48586305 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:07.776346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f278df9-52be-4b9d-91d9-53dc48586305 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:07.776618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f278df9-52be-4b9d-91d9-53dc48586305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:07.777391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f278df9-52be-4b9d-91d9-53dc48586305 None admin] RBAC: Adding query filter params (role.id=231a681c35614ea3aaf3452a8c167c5c, user.id=3bd43df9126c451ea39e84b6fd08e9f4, scope.system=all) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:07.777703 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f278df9-52be-4b9d-91d9-53dc48586305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:07.779248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f278df9-52be-4b9d-91d9-53dc48586305 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:07.792105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 169/330] 10.222.0.60 () {56 vars in 1308 bytes} [Sat Sep 17 04:15:07 2022] GET /identity/v3/role_assignments?user.id=3bd43df9126c451ea39e84b6fd08e9f4&scope.system=all&role.id=231a681c35614ea3aaf3452a8c167c5c => generated 510 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:07.880641 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6f9d4-4e52-4eac-9daa-5149924374ca None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:07.920047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c326d26-2bc9-41cd-a00b-f5bb01586e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:07.920680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c326d26-2bc9-41cd-a00b-f5bb01586e19 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:07.920959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c326d26-2bc9-41cd-a00b-f5bb01586e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:07.921323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c326d26-2bc9-41cd-a00b-f5bb01586e19 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:07.937630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c326d26-2bc9-41cd-a00b-f5bb01586e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6c326d26-2bc9-41cd-a00b-f5bb01586e19 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:07.943156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:07.944091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:07.944902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 162/331] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:07.951053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f278df9-52be-4b9d-91d9-53dc48586305 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:07.982662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:08.061442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f278df9-52be-4b9d-91d9-53dc48586305 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:08.096952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443eaa8-3b08-4ad1-923f-d148db5cea8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:08.097657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443eaa8-3b08-4ad1-923f-d148db5cea8e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:08.097938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443eaa8-3b08-4ad1-923f-d148db5cea8e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:08.098205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443eaa8-3b08-4ad1-923f-d148db5cea8e None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:08.098878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443eaa8-3b08-4ad1-923f-d148db5cea8e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:08.099176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443eaa8-3b08-4ad1-923f-d148db5cea8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:08.100617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443eaa8-3b08-4ad1-923f-d148db5cea8e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:08.109476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 170/332] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:08.115213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c326d26-2bc9-41cd-a00b-f5bb01586e19 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:08.180558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:08.228630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c326d26-2bc9-41cd-a00b-f5bb01586e19 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:08.262693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d0c1c-dc65-44a4-ac4e-4b59affa788c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:08.263680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0c1c-dc65-44a4-ac4e-4b59affa788c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:08.264142 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0c1c-dc65-44a4-ac4e-4b59affa788c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:08.264592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0c1c-dc65-44a4-ac4e-4b59affa788c None admin] PATH_INFO: `/v3/users/30153216ec294123b388e36f177b942b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:08.266045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d0c1c-dc65-44a4-ac4e-4b59affa788c None admin] RBAC: Authorizing `identity:get_user(user_id=30153216ec294123b388e36f177b942b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:08.267718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d0c1c-dc65-44a4-ac4e-4b59affa788c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:08.270007 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 163/333] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:15:08 2022] GET /identity/v3/users/30153216ec294123b388e36f177b942b => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:08.275546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443eaa8-3b08-4ad1-923f-d148db5cea8e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:08.365808 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:08.378507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443eaa8-3b08-4ad1-923f-d148db5cea8e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:08.414105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f3415-8af8-4d33-be98-e5c86db652c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:08.414790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f3415-8af8-4d33-be98-e5c86db652c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:08.415059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f3415-8af8-4d33-be98-e5c86db652c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:08.415364 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f3415-8af8-4d33-be98-e5c86db652c7 None admin] PATH_INFO: `/v3/system/users/30153216ec294123b388e36f177b942b/roles/7dae07bb44b2421baecc0522aaf94e7b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:08.416860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f3415-8af8-4d33-be98-e5c86db652c7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=30153216ec294123b388e36f177b942b, role_id=7dae07bb44b2421baecc0522aaf94e7b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:08.418329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f3415-8af8-4d33-be98-e5c86db652c7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:08.427600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 171/334] 10.222.0.60 () {58 vars in 1334 bytes} [Sat Sep 17 04:15:08 2022] PUT /identity/v3/system/users/30153216ec294123b388e36f177b942b/roles/7dae07bb44b2421baecc0522aaf94e7b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:15:09.776436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b9400-5839-4ac8-a818-fd8ac4299a7e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:09.777457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b9400-5839-4ac8-a818-fd8ac4299a7e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:09.777934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b9400-5839-4ac8-a818-fd8ac4299a7e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:09.779744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 164/335] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:09.785636 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802e416-2fda-49e7-a46c-e9da8ae39dc7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:09.786506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802e416-2fda-49e7-a46c-e9da8ae39dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:09.787018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802e416-2fda-49e7-a46c-e9da8ae39dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:09.808798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d802e416-2fda-49e7-a46c-e9da8ae39dc7 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:09.810407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d802e416-2fda-49e7-a46c-e9da8ae39dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:09.895881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 172/336] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:10.205196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082c282-0a9a-443f-bdd9-9b66ebcdf564 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:10.205502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082c282-0a9a-443f-bdd9-9b66ebcdf564 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:10.205890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082c282-0a9a-443f-bdd9-9b66ebcdf564 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:10.207339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 165/337] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:10.213680 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35a29a-c072-4295-acbc-2d3cdad152bd None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:10.213975 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35a29a-c072-4295-acbc-2d3cdad152bd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:10.214224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35a29a-c072-4295-acbc-2d3cdad152bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:10.237625 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1c35a29a-c072-4295-acbc-2d3cdad152bd None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:10.238796 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1c35a29a-c072-4295-acbc-2d3cdad152bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:10.326749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 173/338] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:10.390763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082c282-0a9a-443f-bdd9-9b66ebcdf564 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:10.521816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082c282-0a9a-443f-bdd9-9b66ebcdf564 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:10.549601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:10.561373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ae27a-d085-4f80-a9b3-a40d3fac4909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:10.562021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ae27a-d085-4f80-a9b3-a40d3fac4909 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:10.562295 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ae27a-d085-4f80-a9b3-a40d3fac4909 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:10.562566 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ae27a-d085-4f80-a9b3-a40d3fac4909 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:10.587222 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ae27a-d085-4f80-a9b3-a40d3fac4909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f89ae27a-d085-4f80-a9b3-a40d3fac4909 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:10.592132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:10.593705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:10.595082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 166/339] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:10.604971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35a29a-c072-4295-acbc-2d3cdad152bd None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:10.633390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:10.681033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49d7fa-5611-4e66-bdbb-be426e14ba3c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:10.681397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49d7fa-5611-4e66-bdbb-be426e14ba3c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:10.681648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49d7fa-5611-4e66-bdbb-be426e14ba3c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:10.683840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 167/340] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:10.689391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299aeeb-70f6-4137-ae59-35cc9e5abcca None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:10.689650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299aeeb-70f6-4137-ae59-35cc9e5abcca None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:10.689930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299aeeb-70f6-4137-ae59-35cc9e5abcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:10.730177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4299aeeb-70f6-4137-ae59-35cc9e5abcca None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:10.731401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4299aeeb-70f6-4137-ae59-35cc9e5abcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:10.792730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35a29a-c072-4295-acbc-2d3cdad152bd None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:10.827276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 168/341] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:10.828147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299aeeb-70f6-4137-ae59-35cc9e5abcca None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:10.874720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c04c3b-61ab-4b8f-bf22-748cc004a94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:10.875352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c04c3b-61ab-4b8f-bf22-748cc004a94d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:10.875632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c04c3b-61ab-4b8f-bf22-748cc004a94d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:10.875899 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c04c3b-61ab-4b8f-bf22-748cc004a94d None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:10.876575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c04c3b-61ab-4b8f-bf22-748cc004a94d None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:10.876875 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c04c3b-61ab-4b8f-bf22-748cc004a94d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:10.878279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c04c3b-61ab-4b8f-bf22-748cc004a94d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:10.896623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 174/342] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:10.903976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c04c3b-61ab-4b8f-bf22-748cc004a94d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:10.959484 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299aeeb-70f6-4137-ae59-35cc9e5abcca None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:10.980005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:10.997955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0167b-24c1-49cd-ac84-501e0b6f349e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:10.998579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0167b-24c1-49cd-ac84-501e0b6f349e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:10.998859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0167b-24c1-49cd-ac84-501e0b6f349e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:10.999134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0167b-24c1-49cd-ac84-501e0b6f349e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:11.013558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f0167b-24c1-49cd-ac84-501e0b6f349e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-24f0167b-24c1-49cd-ac84-501e0b6f349e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:11.018499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:11.019922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:11.021211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 169/343] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:15:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:11.025488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0167b-24c1-49cd-ac84-501e0b6f349e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:11.125781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:11.144659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0167b-24c1-49cd-ac84-501e0b6f349e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:11.180308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6510fe-13e2-40e6-bea5-ca5953c3be25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:11.180953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6510fe-13e2-40e6-bea5-ca5953c3be25 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:11.181264 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6510fe-13e2-40e6-bea5-ca5953c3be25 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:11.181536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6510fe-13e2-40e6-bea5-ca5953c3be25 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:11.182212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6510fe-13e2-40e6-bea5-ca5953c3be25 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:11.182513 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6510fe-13e2-40e6-bea5-ca5953c3be25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:11.183909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6510fe-13e2-40e6-bea5-ca5953c3be25 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:11.192990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 170/344] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:11.198799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6510fe-13e2-40e6-bea5-ca5953c3be25 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:11.200275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c04c3b-61ab-4b8f-bf22-748cc004a94d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:11.243208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:11.296842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb8335-adcb-4b04-8205-82677364ca5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:11.297505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb8335-adcb-4b04-8205-82677364ca5d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:11.297778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb8335-adcb-4b04-8205-82677364ca5d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:11.298046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb8335-adcb-4b04-8205-82677364ca5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:11.305659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6510fe-13e2-40e6-bea5-ca5953c3be25 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:11.341990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0efa2-d5b1-45a8-958f-809a2208c5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:11.342855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0efa2-d5b1-45a8-958f-809a2208c5cb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:11.343273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0efa2-d5b1-45a8-958f-809a2208c5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:11.343681 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0efa2-d5b1-45a8-958f-809a2208c5cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:11.422002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0efa2-d5b1-45a8-958f-809a2208c5cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:11.423782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0efa2-d5b1-45a8-958f-809a2208c5cb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:11.435962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb8335-adcb-4b04-8205-82677364ca5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:11.437368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb8335-adcb-4b04-8205-82677364ca5d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:11.466709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:11.532984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 171/345] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 04:15:11.538761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0efa2-d5b1-45a8-958f-809a2208c5cb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:11.557454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:11.592617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 175/346] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 04:15:11.597862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb8335-adcb-4b04-8205-82677364ca5d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:11.646126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0efa2-d5b1-45a8-958f-809a2208c5cb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:11.687495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4325cbd-bd88-43f8-a6f4-e72d2f8d5604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:11.688063 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4325cbd-bd88-43f8-a6f4-e72d2f8d5604 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:11.688339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4325cbd-bd88-43f8-a6f4-e72d2f8d5604 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:11.688609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4325cbd-bd88-43f8-a6f4-e72d2f8d5604 None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:11.689789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4325cbd-bd88-43f8-a6f4-e72d2f8d5604 None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:11.691252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4325cbd-bd88-43f8-a6f4-e72d2f8d5604 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:11.696001 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 172/347] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:11 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:11.706136 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4325cbd-bd88-43f8-a6f4-e72d2f8d5604 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:11.723692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb8335-adcb-4b04-8205-82677364ca5d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:11.727671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:11.759768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a5e84-972e-40dd-89e3-9b05944c9e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:11.760410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a5e84-972e-40dd-89e3-9b05944c9e45 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:11.760683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a5e84-972e-40dd-89e3-9b05944c9e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:11.760952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a5e84-972e-40dd-89e3-9b05944c9e45 None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:11.762115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a5e84-972e-40dd-89e3-9b05944c9e45 None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:11.763561 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a5e84-972e-40dd-89e3-9b05944c9e45 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:11.765214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 176/348] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:11 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:11.766058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a5e84-972e-40dd-89e3-9b05944c9e45 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:11.816925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:11.884202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a5e84-972e-40dd-89e3-9b05944c9e45 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:11.912920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4325cbd-bd88-43f8-a6f4-e72d2f8d5604 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:11.921638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36164183-8df9-4645-9da7-f7c508544446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:11.922289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36164183-8df9-4645-9da7-f7c508544446 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:11.922565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36164183-8df9-4645-9da7-f7c508544446 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:11.922832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36164183-8df9-4645-9da7-f7c508544446 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:11.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36164183-8df9-4645-9da7-f7c508544446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-36164183-8df9-4645-9da7-f7c508544446 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:11.938791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:11.939757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:11.940599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:11.940599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:11.940599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:11.940599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:11.940599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:11.940599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:11.940599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:11.940599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:11.940599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:11.940599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:11.940599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:11.940599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:11.942818 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 177/349] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:11.943931 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36164183-8df9-4645-9da7-f7c508544446 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:11.981584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1240989-ef1f-455f-8be8-a78cbedff91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:11.982610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1240989-ef1f-455f-8be8-a78cbedff91b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:11.983085 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1240989-ef1f-455f-8be8-a78cbedff91b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:11.983564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1240989-ef1f-455f-8be8-a78cbedff91b None admin] PATH_INFO: `/v3/groups/8d19401d46cc4ff697a7406a98eefcc6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:11.990506 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1240989-ef1f-455f-8be8-a78cbedff91b None admin] RBAC: Authorizing `identity:get_group(group_id=8d19401d46cc4ff697a7406a98eefcc6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:11.992726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1240989-ef1f-455f-8be8-a78cbedff91b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:11.995040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 173/350] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:11 2022] GET /identity/v3/groups/8d19401d46cc4ff697a7406a98eefcc6 => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:11.996455 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde8bf6-b155-4a23-ac36-a0e83d148c2d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:11.996947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde8bf6-b155-4a23-ac36-a0e83d148c2d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:11.997457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde8bf6-b155-4a23-ac36-a0e83d148c2d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:11.998893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 174/351] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:11.999780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde8bf6-b155-4a23-ac36-a0e83d148c2d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:12.078485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36164183-8df9-4645-9da7-f7c508544446 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:12.080243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:12.087072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:12.142527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436f825-db05-4487-a1da-f7f6e80b0a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:12.143258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436f825-db05-4487-a1da-f7f6e80b0a06 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:12.143536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436f825-db05-4487-a1da-f7f6e80b0a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:12.143804 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436f825-db05-4487-a1da-f7f6e80b0a06 None admin] PATH_INFO: `/v3/groups/8d19401d46cc4ff697a7406a98eefcc6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:12.144915 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436f825-db05-4487-a1da-f7f6e80b0a06 None admin] RBAC: Authorizing `identity:get_group(group_id=8d19401d46cc4ff697a7406a98eefcc6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:12.145910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde8bf6-b155-4a23-ac36-a0e83d148c2d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:12.146380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436f825-db05-4487-a1da-f7f6e80b0a06 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:12.148260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 178/352] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:11 2022] GET /identity/v3/groups/8d19401d46cc4ff697a7406a98eefcc6 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:12.150354 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436f825-db05-4487-a1da-f7f6e80b0a06 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:12.196405 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00cef0-cd31-4b72-9584-f99a20dc526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:12.196892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00cef0-cd31-4b72-9584-f99a20dc526b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:12.197351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00cef0-cd31-4b72-9584-f99a20dc526b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:12.197629 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00cef0-cd31-4b72-9584-f99a20dc526b None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:12.198305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00cef0-cd31-4b72-9584-f99a20dc526b None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:12.198607 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00cef0-cd31-4b72-9584-f99a20dc526b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:12.201070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00cef0-cd31-4b72-9584-f99a20dc526b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:12.212850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 175/353] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:12.218365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d95b52-4b9b-4d9e-b904-33c514dc1d7b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:12.218651 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d95b52-4b9b-4d9e-b904-33c514dc1d7b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:12.218920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d95b52-4b9b-4d9e-b904-33c514dc1d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:12.247007 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e1d95b52-4b9b-4d9e-b904-33c514dc1d7b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:12.249195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e1d95b52-4b9b-4d9e-b904-33c514dc1d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:12.293322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436f825-db05-4487-a1da-f7f6e80b0a06 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:12.331448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5c380-5659-40ff-99d0-568e536c77d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:12.332073 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5c380-5659-40ff-99d0-568e536c77d2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:12.332346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5c380-5659-40ff-99d0-568e536c77d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:12.332631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5c380-5659-40ff-99d0-568e536c77d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:12.333388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5c380-5659-40ff-99d0-568e536c77d2 None admin] RBAC: Adding query filter params (group.id=8d19401d46cc4ff697a7406a98eefcc6, role.id=84212031716c4cf8a5083824fe47dbb7, scope.project.id=d7203986da504ab1ba9f76d7f64f7dce) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:12.333694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5c380-5659-40ff-99d0-568e536c77d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:12.335161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5c380-5659-40ff-99d0-568e536c77d2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:12.341485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 179/354] 10.222.0.60 () {56 vars in 1376 bytes} [Sat Sep 17 04:15:12 2022] GET /identity/v3/role_assignments?group.id=8d19401d46cc4ff697a7406a98eefcc6&scope.project.id=d7203986da504ab1ba9f76d7f64f7dce&role.id=84212031716c4cf8a5083824fe47dbb7 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:12.342344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5c380-5659-40ff-99d0-568e536c77d2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:12.348639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 176/355] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:12.350259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d95b52-4b9b-4d9e-b904-33c514dc1d7b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:12.426733 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:12.441545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:12.491145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d95b52-4b9b-4d9e-b904-33c514dc1d7b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:12.557482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce517d0-84c8-4116-b266-d96908bec653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:12.558951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce517d0-84c8-4116-b266-d96908bec653 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:12.559225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce517d0-84c8-4116-b266-d96908bec653 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:12.559492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce517d0-84c8-4116-b266-d96908bec653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:12.564838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5c380-5659-40ff-99d0-568e536c77d2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:12.637207 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971d5bb-a59e-49bf-b0fc-ce104fef2522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:12.637849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971d5bb-a59e-49bf-b0fc-ce104fef2522 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:12.638126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971d5bb-a59e-49bf-b0fc-ce104fef2522 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:12.638412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971d5bb-a59e-49bf-b0fc-ce104fef2522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:12.639198 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971d5bb-a59e-49bf-b0fc-ce104fef2522 None admin] RBAC: Adding query filter params (group.id=8d19401d46cc4ff697a7406a98eefcc6, role.id=231a681c35614ea3aaf3452a8c167c5c, scope.project.id=d7203986da504ab1ba9f76d7f64f7dce) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:12.639507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971d5bb-a59e-49bf-b0fc-ce104fef2522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:12.640989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971d5bb-a59e-49bf-b0fc-ce104fef2522 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:12.663505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 180/356] 10.222.0.60 () {56 vars in 1376 bytes} [Sat Sep 17 04:15:12 2022] GET /identity/v3/role_assignments?group.id=8d19401d46cc4ff697a7406a98eefcc6&scope.project.id=d7203986da504ab1ba9f76d7f64f7dce&role.id=231a681c35614ea3aaf3452a8c167c5c => generated 260 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:12.670243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0a4cd-5c2c-4fe7-97d9-416184458af0 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:12.672988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0a4cd-5c2c-4fe7-97d9-416184458af0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:12.673344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0a4cd-5c2c-4fe7-97d9-416184458af0 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:12.674858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 181/357] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:12.693151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e65ef-f92d-4f6f-bb0e-56c179eeeb56 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:12.693471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e65ef-f92d-4f6f-bb0e-56c179eeeb56 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:12.693668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e65ef-f92d-4f6f-bb0e-56c179eeeb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:12.717638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a75e65ef-f92d-4f6f-bb0e-56c179eeeb56 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:12.718856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a75e65ef-f92d-4f6f-bb0e-56c179eeeb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:12.726082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce517d0-84c8-4116-b266-d96908bec653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:12.728213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce517d0-84c8-4116-b266-d96908bec653 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:12.797643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:12.808539 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 182/358] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:12.816716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e65ef-f92d-4f6f-bb0e-56c179eeeb56 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:12.844243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 177/359] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:12.851279 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce517d0-84c8-4116-b266-d96908bec653 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:12.876781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:12.961194 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e65ef-f92d-4f6f-bb0e-56c179eeeb56 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:13.019480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce517d0-84c8-4116-b266-d96908bec653 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:13.035753 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7e178-b273-4eef-bc56-b4c73aec25d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:13.037367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7e178-b273-4eef-bc56-b4c73aec25d7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:13.038205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7e178-b273-4eef-bc56-b4c73aec25d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:13.038934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7e178-b273-4eef-bc56-b4c73aec25d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:13.058490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7e178-b273-4eef-bc56-b4c73aec25d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e2d7e178-b273-4eef-bc56-b4c73aec25d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:13.064351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:13.065267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:13.066135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:13.066135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:13.066135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:13.066135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:13.066135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:13.066135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:13.066135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:13.066135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:13.066135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:13.066135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:13.066135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 04:15:13.066135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:13.071231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 183/360] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:13.072787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13b040-38ab-4e29-816a-c37a5d395915 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:13.073889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13b040-38ab-4e29-816a-c37a5d395915 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:13.074308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13b040-38ab-4e29-816a-c37a5d395915 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:13.074780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e9c88-8ebe-47d8-a045-8886862fc290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:13.075765 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e9c88-8ebe-47d8-a045-8886862fc290 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:13.076263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e9c88-8ebe-47d8-a045-8886862fc290 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:13.078075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e9c88-8ebe-47d8-a045-8886862fc290 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:13.078295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 184/361] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:13.081784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13b040-38ab-4e29-816a-c37a5d395915 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:13.081927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e9c88-8ebe-47d8-a045-8886862fc290 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:13.084105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e9c88-8ebe-47d8-a045-8886862fc290 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:13.086396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 178/362] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:12 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:13.092008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e9c88-8ebe-47d8-a045-8886862fc290 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:13.136873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:13.206132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:13.225560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13b040-38ab-4e29-816a-c37a5d395915 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:13.243248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e9c88-8ebe-47d8-a045-8886862fc290 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:13.272928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4a4f1-9b74-4e5e-851a-fe851e0c076a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:13.273988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4a4f1-9b74-4e5e-851a-fe851e0c076a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:13.274272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4a4f1-9b74-4e5e-851a-fe851e0c076a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:13.274836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4a4f1-9b74-4e5e-851a-fe851e0c076a None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:13.275520 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4a4f1-9b74-4e5e-851a-fe851e0c076a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:13.276062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4a4f1-9b74-4e5e-851a-fe851e0c076a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:13.278262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4a4f1-9b74-4e5e-851a-fe851e0c076a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:13.282391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4cf0-6cac-4321-9478-19ba7ccb1d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:13.283099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4cf0-6cac-4321-9478-19ba7ccb1d72 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:13.283378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4cf0-6cac-4321-9478-19ba7ccb1d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:13.283652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4cf0-6cac-4321-9478-19ba7ccb1d72 None admin] PATH_INFO: `/v3/groups/8d19401d46cc4ff697a7406a98eefcc6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:13.284766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e4cf0-6cac-4321-9478-19ba7ccb1d72 None admin] RBAC: Authorizing `identity:get_group(group_id=8d19401d46cc4ff697a7406a98eefcc6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:13.286261 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e4cf0-6cac-4321-9478-19ba7ccb1d72 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:13.287680 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 185/363] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:13.288192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 179/364] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:13 2022] GET /identity/v3/groups/8d19401d46cc4ff697a7406a98eefcc6 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:13.289129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df430775-ab19-4703-8924-09ad39d50b80 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:13.289254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4cf0-6cac-4321-9478-19ba7ccb1d72 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:13.289343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df430775-ab19-4703-8924-09ad39d50b80 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:13.289722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df430775-ab19-4703-8924-09ad39d50b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:13.312666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-df430775-ab19-4703-8924-09ad39d50b80 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:13.314329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-df430775-ab19-4703-8924-09ad39d50b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:13.406528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 186/365] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:13.407706 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df430775-ab19-4703-8924-09ad39d50b80 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:13.408379 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:13.412224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4cf0-6cac-4321-9478-19ba7ccb1d72 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:13.453018 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b55c7-dd85-4b2e-b45c-7d5d612dc7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:13.453706 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b55c7-dd85-4b2e-b45c-7d5d612dc7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:13.453983 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b55c7-dd85-4b2e-b45c-7d5d612dc7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:13.454248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b55c7-dd85-4b2e-b45c-7d5d612dc7c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:13.468418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b55c7-dd85-4b2e-b45c-7d5d612dc7c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-202b55c7-dd85-4b2e-b45c-7d5d612dc7c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:13.478984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:13.481182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:13.483180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:13.483180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:13.483180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:13.483180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:13.483180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:13.483180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:13.483180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:13.483180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:13.483180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:13.483180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:13.483180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:13.483180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:13.484642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 180/366] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:15:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:13.489423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b55c7-dd85-4b2e-b45c-7d5d612dc7c3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:13.506549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:13.532958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df430775-ab19-4703-8924-09ad39d50b80 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:13.583785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbce53c-b38b-4fcd-b6e9-0f3650161727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:13.584358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbce53c-b38b-4fcd-b6e9-0f3650161727 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:13.584631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbce53c-b38b-4fcd-b6e9-0f3650161727 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:13.584898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbce53c-b38b-4fcd-b6e9-0f3650161727 None admin] PATH_INFO: `/v3/users/30153216ec294123b388e36f177b942b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:13.586071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbce53c-b38b-4fcd-b6e9-0f3650161727 None admin] RBAC: Authorizing `identity:get_user(user_id=30153216ec294123b388e36f177b942b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:13.588737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbce53c-b38b-4fcd-b6e9-0f3650161727 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:13.591481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 187/367] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:15:13 2022] GET /identity/v3/users/30153216ec294123b388e36f177b942b => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:13.592410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbce53c-b38b-4fcd-b6e9-0f3650161727 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:13.646192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b55c7-dd85-4b2e-b45c-7d5d612dc7c3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:13.720605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbce53c-b38b-4fcd-b6e9-0f3650161727 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:13.733131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb862a-e294-49d7-af80-1ce2a6eb9d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:13.733953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb862a-e294-49d7-af80-1ce2a6eb9d3a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:13.734298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb862a-e294-49d7-af80-1ce2a6eb9d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:13.734686 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb862a-e294-49d7-af80-1ce2a6eb9d3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:13.735527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb862a-e294-49d7-af80-1ce2a6eb9d3a None admin] RBAC: Adding query filter params (group.id=8d19401d46cc4ff697a7406a98eefcc6, role.id=231a681c35614ea3aaf3452a8c167c5c, scope.project.id=96874534e5534d3699fb67c58215c104) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:13.735932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb862a-e294-49d7-af80-1ce2a6eb9d3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:13.737712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb862a-e294-49d7-af80-1ce2a6eb9d3a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:13.750178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 181/368] 10.222.0.60 () {56 vars in 1376 bytes} [Sat Sep 17 04:15:13 2022] GET /identity/v3/role_assignments?group.id=8d19401d46cc4ff697a7406a98eefcc6&scope.project.id=96874534e5534d3699fb67c58215c104&role.id=231a681c35614ea3aaf3452a8c167c5c => generated 260 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:13.762763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb862a-e294-49d7-af80-1ce2a6eb9d3a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:13.768841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af19989-d974-4b4a-bb68-90086b696d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:13.769818 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af19989-d974-4b4a-bb68-90086b696d7c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:13.770380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af19989-d974-4b4a-bb68-90086b696d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:13.770842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af19989-d974-4b4a-bb68-90086b696d7c None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:13.771707 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af19989-d974-4b4a-bb68-90086b696d7c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:13.772787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af19989-d974-4b4a-bb68-90086b696d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:13.775118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af19989-d974-4b4a-bb68-90086b696d7c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:13.792539 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 188/369] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:13.793907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af19989-d974-4b4a-bb68-90086b696d7c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:13.811676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:13.821542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:13.924525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af19989-d974-4b4a-bb68-90086b696d7c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:13.964508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de941bde-a21b-4bee-8736-cbe2dd0890ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:13.965416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de941bde-a21b-4bee-8736-cbe2dd0890ce None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:13.965802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de941bde-a21b-4bee-8736-cbe2dd0890ce None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:13.966745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de941bde-a21b-4bee-8736-cbe2dd0890ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:13.980945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de941bde-a21b-4bee-8736-cbe2dd0890ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-de941bde-a21b-4bee-8736-cbe2dd0890ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:13.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:13.988309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 189/370] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:15:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:13.989118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de941bde-a21b-4bee-8736-cbe2dd0890ce None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:14.003115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb862a-e294-49d7-af80-1ce2a6eb9d3a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:14.052251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bda76-d207-46d6-a771-735c66eba4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:14.053632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bda76-d207-46d6-a771-735c66eba4f3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:14.054085 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bda76-d207-46d6-a771-735c66eba4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:14.054526 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bda76-d207-46d6-a771-735c66eba4f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:14.055614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687bda76-d207-46d6-a771-735c66eba4f3 None admin] RBAC: Adding query filter params (role.id=7dae07bb44b2421baecc0522aaf94e7b, user.id=30153216ec294123b388e36f177b942b, scope.system=all) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:14.056169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687bda76-d207-46d6-a771-735c66eba4f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:14.057947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687bda76-d207-46d6-a771-735c66eba4f3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:14.072183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 182/371] 10.222.0.60 () {56 vars in 1308 bytes} [Sat Sep 17 04:15:13 2022] GET /identity/v3/role_assignments?user.id=30153216ec294123b388e36f177b942b&scope.system=all&role.id=7dae07bb44b2421baecc0522aaf94e7b => generated 510 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:14.073413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bda76-d207-46d6-a771-735c66eba4f3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:14.078233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:14.122244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de941bde-a21b-4bee-8736-cbe2dd0890ce None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:14.180130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5847a-8fa1-4e84-ba63-7c14e56913f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:14.180955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5847a-8fa1-4e84-ba63-7c14e56913f4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:14.181275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5847a-8fa1-4e84-ba63-7c14e56913f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:14.181546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5847a-8fa1-4e84-ba63-7c14e56913f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:14.277964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5847a-8fa1-4e84-ba63-7c14e56913f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:14.279617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5847a-8fa1-4e84-ba63-7c14e56913f4 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:14.322888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:14.365413 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:14.369422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bda76-d207-46d6-a771-735c66eba4f3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:14.398951 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 190/372] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:14.405388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5847a-8fa1-4e84-ba63-7c14e56913f4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:14.503533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c887c-699d-4e1f-8d2d-e29d9f015c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:14.504472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c887c-699d-4e1f-8d2d-e29d9f015c4b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:14.504947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c887c-699d-4e1f-8d2d-e29d9f015c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:14.505471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c887c-699d-4e1f-8d2d-e29d9f015c4b None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:14.506643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c887c-699d-4e1f-8d2d-e29d9f015c4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:14.507247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c887c-699d-4e1f-8d2d-e29d9f015c4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:14.509744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c887c-699d-4e1f-8d2d-e29d9f015c4b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:14.534199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5847a-8fa1-4e84-ba63-7c14e56913f4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:14.538964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 183/373] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:14.546196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c887c-699d-4e1f-8d2d-e29d9f015c4b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:14.574387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85936421-99da-42fb-ab05-8699a9f04f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:14.575048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85936421-99da-42fb-ab05-8699a9f04f84 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:14.575336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85936421-99da-42fb-ab05-8699a9f04f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:14.575616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85936421-99da-42fb-ab05-8699a9f04f84 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:14.576771 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85936421-99da-42fb-ab05-8699a9f04f84 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:14.578526 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85936421-99da-42fb-ab05-8699a9f04f84 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:14.580486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 191/374] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:14 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:14.587075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85936421-99da-42fb-ab05-8699a9f04f84 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:14.636720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:14.711893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85936421-99da-42fb-ab05-8699a9f04f84 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:14.749314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c887c-699d-4e1f-8d2d-e29d9f015c4b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:14.749778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41068480-da5b-4ec0-81ff-c68a9d0dd1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:14.750611 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41068480-da5b-4ec0-81ff-c68a9d0dd1ed None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:14.751042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41068480-da5b-4ec0-81ff-c68a9d0dd1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:14.751451 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41068480-da5b-4ec0-81ff-c68a9d0dd1ed None admin] PATH_INFO: `/v3/groups/8d19401d46cc4ff697a7406a98eefcc6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:14.752729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41068480-da5b-4ec0-81ff-c68a9d0dd1ed None admin] RBAC: Authorizing `identity:get_group(group_id=8d19401d46cc4ff697a7406a98eefcc6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:14.754413 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41068480-da5b-4ec0-81ff-c68a9d0dd1ed None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:14.756498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 192/375] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:14 2022] GET /identity/v3/groups/8d19401d46cc4ff697a7406a98eefcc6 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:14.762546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41068480-da5b-4ec0-81ff-c68a9d0dd1ed None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:14.822867 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7911cac-98ec-4a37-87d9-654b35700c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:14.824250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7911cac-98ec-4a37-87d9-654b35700c8d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:14.824639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7911cac-98ec-4a37-87d9-654b35700c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:14.825018 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7911cac-98ec-4a37-87d9-654b35700c8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:14.855299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:14.870759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41068480-da5b-4ec0-81ff-c68a9d0dd1ed None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:14.891070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:14.904991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059242e0-91f4-44c1-9452-dcfaa39b6190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:14.905640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059242e0-91f4-44c1-9452-dcfaa39b6190 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:14.905923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059242e0-91f4-44c1-9452-dcfaa39b6190 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:14.906198 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059242e0-91f4-44c1-9452-dcfaa39b6190 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:14.906934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059242e0-91f4-44c1-9452-dcfaa39b6190 None admin] RBAC: Adding query filter params (group.id=8d19401d46cc4ff697a7406a98eefcc6, role.id=84212031716c4cf8a5083824fe47dbb7, scope.project.id=96874534e5534d3699fb67c58215c104) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:14.907579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059242e0-91f4-44c1-9452-dcfaa39b6190 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:14.909310 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059242e0-91f4-44c1-9452-dcfaa39b6190 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:14.915925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 193/376] 10.222.0.60 () {56 vars in 1376 bytes} [Sat Sep 17 04:15:14 2022] GET /identity/v3/role_assignments?group.id=8d19401d46cc4ff697a7406a98eefcc6&scope.project.id=96874534e5534d3699fb67c58215c104&role.id=84212031716c4cf8a5083824fe47dbb7 => generated 260 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:14.945574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7911cac-98ec-4a37-87d9-654b35700c8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:14.947283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7911cac-98ec-4a37-87d9-654b35700c8d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:15.084285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 184/377] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:15.090861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059242e0-91f4-44c1-9452-dcfaa39b6190 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:15.211211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059242e0-91f4-44c1-9452-dcfaa39b6190 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:15.247417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da14939-9df9-48d9-89ff-c8471d63c86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:15.248078 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da14939-9df9-48d9-89ff-c8471d63c86c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:15.248359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da14939-9df9-48d9-89ff-c8471d63c86c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:15.248632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da14939-9df9-48d9-89ff-c8471d63c86c None admin] PATH_INFO: `/v3/projects/7fd2fb11102f467d836ef30d6efdd922` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:15.251350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da14939-9df9-48d9-89ff-c8471d63c86c None admin] RBAC: Authorizing `identity:get_project(project_id=7fd2fb11102f467d836ef30d6efdd922)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:15.252801 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da14939-9df9-48d9-89ff-c8471d63c86c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:15.254836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 194/378] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:15 2022] GET /identity/v3/projects/7fd2fb11102f467d836ef30d6efdd922 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:15.262477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7911cac-98ec-4a37-87d9-654b35700c8d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:15.320560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:15.390750 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7911cac-98ec-4a37-87d9-654b35700c8d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:15.441448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870abf40-5a21-4a6a-b0b2-0b9f282a1009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:15.442092 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870abf40-5a21-4a6a-b0b2-0b9f282a1009 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:15.442365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870abf40-5a21-4a6a-b0b2-0b9f282a1009 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:15.442629 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870abf40-5a21-4a6a-b0b2-0b9f282a1009 None admin] PATH_INFO: `/v3/groups/13cad0fff80a4dfdba252827031bb0f8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:15.449826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870abf40-5a21-4a6a-b0b2-0b9f282a1009 None admin] RBAC: Authorizing `identity:get_group(group_id=13cad0fff80a4dfdba252827031bb0f8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:15.452013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870abf40-5a21-4a6a-b0b2-0b9f282a1009 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:15.454930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 185/379] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:15 2022] GET /identity/v3/groups/13cad0fff80a4dfdba252827031bb0f8 => generated 229 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:15.460980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da14939-9df9-48d9-89ff-c8471d63c86c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:15.533788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:15.634751 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da14939-9df9-48d9-89ff-c8471d63c86c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:15.716455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b2e22-c754-4f6e-9260-009a5d2f6f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:15.717361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b2e22-c754-4f6e-9260-009a5d2f6f08 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:15.717743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b2e22-c754-4f6e-9260-009a5d2f6f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:15.718111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b2e22-c754-4f6e-9260-009a5d2f6f08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:15.719184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b2e22-c754-4f6e-9260-009a5d2f6f08 None admin] RBAC: Adding query filter params (group.id=13cad0fff80a4dfdba252827031bb0f8, role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=7fd2fb11102f467d836ef30d6efdd922) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:15.719616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b2e22-c754-4f6e-9260-009a5d2f6f08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:15.721792 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b2e22-c754-4f6e-9260-009a5d2f6f08 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:15.736125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 195/380] 10.222.0.60 () {56 vars in 1376 bytes} [Sat Sep 17 04:15:15 2022] GET /identity/v3/role_assignments?group.id=13cad0fff80a4dfdba252827031bb0f8&scope.project.id=7fd2fb11102f467d836ef30d6efdd922&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 260 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:15.815000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7ece6-86ee-41d5-a0cf-66ff60b3ec11 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:15.815379 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7ece6-86ee-41d5-a0cf-66ff60b3ec11 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:15.815749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7ece6-86ee-41d5-a0cf-66ff60b3ec11 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:15.817694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 186/381] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:15.823826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294deb-2036-45ad-a531-3ddfc363d84e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:15.824265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294deb-2036-45ad-a531-3ddfc363d84e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:15.824636 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294deb-2036-45ad-a531-3ddfc363d84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:15.845723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40195e-d130-472a-930c-c0f83709fcc3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:15.846063 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40195e-d130-472a-930c-c0f83709fcc3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:15.846263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40195e-d130-472a-930c-c0f83709fcc3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:15.847999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 187/382] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:15.854158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fede94-2ea9-4cca-8853-7c34f4d334f4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:15.854585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fede94-2ea9-4cca-8853-7c34f4d334f4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:15.854773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fede94-2ea9-4cca-8853-7c34f4d334f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:15.860577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-15294deb-2036-45ad-a531-3ddfc363d84e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:15.862470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-15294deb-2036-45ad-a531-3ddfc363d84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:15.896070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d3fede94-2ea9-4cca-8853-7c34f4d334f4 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:15.897312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d3fede94-2ea9-4cca-8853-7c34f4d334f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:15.986924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 188/383] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:15.995678 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 196/384] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:16.305951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fede94-2ea9-4cca-8853-7c34f4d334f4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:16.354038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:16.366289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15294deb-2036-45ad-a531-3ddfc363d84e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:16.419841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:16.428410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fede94-2ea9-4cca-8853-7c34f4d334f4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:16.474725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7628435-1b5e-429e-b23a-7361a6c69cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:16.475369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7628435-1b5e-429e-b23a-7361a6c69cad None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:16.475645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7628435-1b5e-429e-b23a-7361a6c69cad None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:16.475915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7628435-1b5e-429e-b23a-7361a6c69cad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:16.502000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7628435-1b5e-429e-b23a-7361a6c69cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d7628435-1b5e-429e-b23a-7361a6c69cad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:16.506879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:16.507761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:16.508530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 189/385] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:15:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:16.511210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15294deb-2036-45ad-a531-3ddfc363d84e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:16.518108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7628435-1b5e-429e-b23a-7361a6c69cad None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:16.550259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a57fb2-bdc6-4bec-a100-b785548fb0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:16.550904 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a57fb2-bdc6-4bec-a100-b785548fb0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:16.551182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a57fb2-bdc6-4bec-a100-b785548fb0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:16.551455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a57fb2-bdc6-4bec-a100-b785548fb0a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:16.563397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a57fb2-bdc6-4bec-a100-b785548fb0a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-16a57fb2-bdc6-4bec-a100-b785548fb0a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:16.568479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:16.569374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:16.570150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:16.570150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:16.570150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:16.570150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:16.570150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:16.570150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:16.570150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:16.570150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:16.570150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:16.570150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:16.570150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:16.570150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:16.572316 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 197/386] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:16.578245 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a57fb2-bdc6-4bec-a100-b785548fb0a0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:16.658178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7628435-1b5e-429e-b23a-7361a6c69cad None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:16.660262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:16.703013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97598de-b403-4c9a-aab2-b01ad4056523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:16.704076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97598de-b403-4c9a-aab2-b01ad4056523 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:16.704440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97598de-b403-4c9a-aab2-b01ad4056523 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:16.704797 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97598de-b403-4c9a-aab2-b01ad4056523 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:16.705733 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97598de-b403-4c9a-aab2-b01ad4056523 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:16.706503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97598de-b403-4c9a-aab2-b01ad4056523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:16.708266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97598de-b403-4c9a-aab2-b01ad4056523 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:16.709238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a57fb2-bdc6-4bec-a100-b785548fb0a0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:16.711721 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:16.725947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 190/387] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:16.731272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97598de-b403-4c9a-aab2-b01ad4056523 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:16.755165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de5c31-8c7e-4207-8cc1-6b6e8ea58238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:16.756058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de5c31-8c7e-4207-8cc1-6b6e8ea58238 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:16.756473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de5c31-8c7e-4207-8cc1-6b6e8ea58238 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:16.756881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de5c31-8c7e-4207-8cc1-6b6e8ea58238 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:16.757714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de5c31-8c7e-4207-8cc1-6b6e8ea58238 None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:16.758232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de5c31-8c7e-4207-8cc1-6b6e8ea58238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:16.759925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de5c31-8c7e-4207-8cc1-6b6e8ea58238 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:16.775342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 198/388] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:16.776899 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de5c31-8c7e-4207-8cc1-6b6e8ea58238 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:16.855440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97598de-b403-4c9a-aab2-b01ad4056523 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:16.894015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee885feb-c2a9-44a9-a2b2-75c80515e0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:16.895037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee885feb-c2a9-44a9-a2b2-75c80515e0fe None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:16.895514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee885feb-c2a9-44a9-a2b2-75c80515e0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:16.895980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee885feb-c2a9-44a9-a2b2-75c80515e0fe None admin] PATH_INFO: `/v3/groups/8d19401d46cc4ff697a7406a98eefcc6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:16.900227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de5c31-8c7e-4207-8cc1-6b6e8ea58238 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:16.900842 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee885feb-c2a9-44a9-a2b2-75c80515e0fe None admin] RBAC: Authorizing `identity:get_group(group_id=8d19401d46cc4ff697a7406a98eefcc6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:16.902554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee885feb-c2a9-44a9-a2b2-75c80515e0fe None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:16.904774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 191/389] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:16 2022] GET /identity/v3/groups/8d19401d46cc4ff697a7406a98eefcc6 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:16.909933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee885feb-c2a9-44a9-a2b2-75c80515e0fe None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:16.937830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf7b9f-aa56-46c9-a43a-be40ba4a461d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:16.938480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf7b9f-aa56-46c9-a43a-be40ba4a461d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:16.938753 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf7b9f-aa56-46c9-a43a-be40ba4a461d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:16.939019 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf7b9f-aa56-46c9-a43a-be40ba4a461d None admin] PATH_INFO: `/v3/groups/8d19401d46cc4ff697a7406a98eefcc6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:16.940326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf7b9f-aa56-46c9-a43a-be40ba4a461d None admin] RBAC: Authorizing `identity:get_group(group_id=8d19401d46cc4ff697a7406a98eefcc6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:16.942000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf7b9f-aa56-46c9-a43a-be40ba4a461d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:16.944127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 199/390] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:16 2022] GET /identity/v3/groups/8d19401d46cc4ff697a7406a98eefcc6 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:16.954648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf7b9f-aa56-46c9-a43a-be40ba4a461d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:16.961580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:16.985348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:17.070329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf7b9f-aa56-46c9-a43a-be40ba4a461d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:17.109016 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9ac10-24af-4397-b4e4-9eeb53112e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:17.109589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9ac10-24af-4397-b4e4-9eeb53112e2f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:17.109862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9ac10-24af-4397-b4e4-9eeb53112e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:17.110128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9ac10-24af-4397-b4e4-9eeb53112e2f None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:17.111262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9ac10-24af-4397-b4e4-9eeb53112e2f None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:17.113174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9ac10-24af-4397-b4e4-9eeb53112e2f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:17.115139 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 200/391] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:16 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:17.124988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9ac10-24af-4397-b4e4-9eeb53112e2f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:17.134141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee885feb-c2a9-44a9-a2b2-75c80515e0fe None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:17.181844 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b154a-0588-4419-9e81-909d2ec3af04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:17.182542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b154a-0588-4419-9e81-909d2ec3af04 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:17.182822 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b154a-0588-4419-9e81-909d2ec3af04 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:17.183091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b154a-0588-4419-9e81-909d2ec3af04 None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:17.189110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b154a-0588-4419-9e81-909d2ec3af04 None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:17.189110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b154a-0588-4419-9e81-909d2ec3af04 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:17.189349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 192/392] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:16 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:17.194821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b154a-0588-4419-9e81-909d2ec3af04 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:17.310217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:17.329271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9ac10-24af-4397-b4e4-9eeb53112e2f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:17.369639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:17.378773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b154a-0588-4419-9e81-909d2ec3af04 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:17.390676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a0e65-2025-4e7b-94cc-21a9b84d113b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:17.391387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a0e65-2025-4e7b-94cc-21a9b84d113b None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:17.391748 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a0e65-2025-4e7b-94cc-21a9b84d113b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:17.392210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a0e65-2025-4e7b-94cc-21a9b84d113b None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce/groups/8d19401d46cc4ff697a7406a98eefcc6/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:17.395884 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a0e65-2025-4e7b-94cc-21a9b84d113b None admin] RBAC: Authorizing `identity:create_grant(project_id=d7203986da504ab1ba9f76d7f64f7dce, group_id=8d19401d46cc4ff697a7406a98eefcc6, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:17.397349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a0e65-2025-4e7b-94cc-21a9b84d113b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:17.421008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 201/393] 10.222.0.60 () {58 vars in 1442 bytes} [Sat Sep 17 04:15:17 2022] PUT /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce/groups/8d19401d46cc4ff697a7406a98eefcc6/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:15:17.443987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268baed0-36c2-4ddf-b3b5-534451d99bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:17.444804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268baed0-36c2-4ddf-b3b5-534451d99bb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:17.445102 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268baed0-36c2-4ddf-b3b5-534451d99bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:17.445373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268baed0-36c2-4ddf-b3b5-534451d99bb7 None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce/groups/8d19401d46cc4ff697a7406a98eefcc6/roles/84212031716c4cf8a5083824fe47dbb7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:17.447764 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268baed0-36c2-4ddf-b3b5-534451d99bb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7203986da504ab1ba9f76d7f64f7dce, group_id=8d19401d46cc4ff697a7406a98eefcc6, role_id=84212031716c4cf8a5083824fe47dbb7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:17.452887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268baed0-36c2-4ddf-b3b5-534451d99bb7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:17.475299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 193/394] 10.222.0.60 () {58 vars in 1442 bytes} [Sat Sep 17 04:15:17 2022] PUT /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce/groups/8d19401d46cc4ff697a7406a98eefcc6/roles/84212031716c4cf8a5083824fe47dbb7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:15:17.676233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464de68-aa18-4c6b-9d4f-859e9f9e671b None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:17.676770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464de68-aa18-4c6b-9d4f-859e9f9e671b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:17.677272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464de68-aa18-4c6b-9d4f-859e9f9e671b None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:17.678797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 202/395] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:15:17.684766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478d280-3db6-4a57-a649-fa94f4cec3bc None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:17.685365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478d280-3db6-4a57-a649-fa94f4cec3bc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:17.685801 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478d280-3db6-4a57-a649-fa94f4cec3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:17.709780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4478d280-3db6-4a57-a649-fa94f4cec3bc None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:17.711214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4478d280-3db6-4a57-a649-fa94f4cec3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:17.783776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 194/396] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:18.078301 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464de68-aa18-4c6b-9d4f-859e9f9e671b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:18.193249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464de68-aa18-4c6b-9d4f-859e9f9e671b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:18.205873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ad86e-da9d-4caf-aba9-70138a5f722c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:18.206496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ad86e-da9d-4caf-aba9-70138a5f722c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:18.206985 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ad86e-da9d-4caf-aba9-70138a5f722c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:18.210000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 195/397] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:18.214658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267f0e8-630a-41a7-94df-411057e7eee3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:18.215217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267f0e8-630a-41a7-94df-411057e7eee3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:18.215684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267f0e8-630a-41a7-94df-411057e7eee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:18.235635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86300ac0-09c8-46a4-89dc-4cb6d5f01324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:18.236637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86300ac0-09c8-46a4-89dc-4cb6d5f01324 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:18.237249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86300ac0-09c8-46a4-89dc-4cb6d5f01324 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:18.237713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86300ac0-09c8-46a4-89dc-4cb6d5f01324 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:18.239442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0267f0e8-630a-41a7-94df-411057e7eee3 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:18.240711 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0267f0e8-630a-41a7-94df-411057e7eee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:18.262050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86300ac0-09c8-46a4-89dc-4cb6d5f01324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-86300ac0-09c8-46a4-89dc-4cb6d5f01324 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:18.267923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:18.268791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:18.269607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:18.269607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:18.269607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:18.269607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:18.269607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:18.269607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:18.269607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:18.269607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:18.269607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:18.269607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:18.269607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:18.269607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:18.271656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 203/398] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:18.278384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86300ac0-09c8-46a4-89dc-4cb6d5f01324 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:18.304040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:18.353705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 196/399] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:18.422272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86300ac0-09c8-46a4-89dc-4cb6d5f01324 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:18.456630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90259fb3-6537-4650-8a91-2afdbe6a6522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:18.457281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90259fb3-6537-4650-8a91-2afdbe6a6522 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:18.457556 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90259fb3-6537-4650-8a91-2afdbe6a6522 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:18.457822 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90259fb3-6537-4650-8a91-2afdbe6a6522 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:18.458488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90259fb3-6537-4650-8a91-2afdbe6a6522 None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:18.458786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90259fb3-6537-4650-8a91-2afdbe6a6522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:18.460285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90259fb3-6537-4650-8a91-2afdbe6a6522 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:18.470005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 204/400] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:18.478432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267f0e8-630a-41a7-94df-411057e7eee3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:18.499944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:18.608920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267f0e8-630a-41a7-94df-411057e7eee3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:18.653408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860af116-e4eb-4922-b9df-92cc9376eeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:18.654052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860af116-e4eb-4922-b9df-92cc9376eeb2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:18.654326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860af116-e4eb-4922-b9df-92cc9376eeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:18.654594 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860af116-e4eb-4922-b9df-92cc9376eeb2 None admin] PATH_INFO: `/v3/groups/8d19401d46cc4ff697a7406a98eefcc6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:18.655687 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860af116-e4eb-4922-b9df-92cc9376eeb2 None admin] RBAC: Authorizing `identity:get_group(group_id=8d19401d46cc4ff697a7406a98eefcc6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:18.657233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860af116-e4eb-4922-b9df-92cc9376eeb2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:18.659390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 197/401] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:18 2022] GET /identity/v3/groups/8d19401d46cc4ff697a7406a98eefcc6 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:18.671819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90259fb3-6537-4650-8a91-2afdbe6a6522 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:18.753010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:18.786588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90259fb3-6537-4650-8a91-2afdbe6a6522 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:18.823057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26c6a7-c197-40b9-9fe7-b57623929bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:18.823707 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26c6a7-c197-40b9-9fe7-b57623929bcd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:18.823982 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26c6a7-c197-40b9-9fe7-b57623929bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:18.824252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26c6a7-c197-40b9-9fe7-b57623929bcd None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:18.825395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26c6a7-c197-40b9-9fe7-b57623929bcd None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:18.826894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26c6a7-c197-40b9-9fe7-b57623929bcd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:18.828850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 205/402] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:18 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:18.834653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860af116-e4eb-4922-b9df-92cc9376eeb2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:18.926406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:18.938635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26c6a7-c197-40b9-9fe7-b57623929bcd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:18.951562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860af116-e4eb-4922-b9df-92cc9376eeb2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:18.996235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005dbe3e-1e18-417b-9ea8-ddd208089838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:18.997297 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005dbe3e-1e18-417b-9ea8-ddd208089838 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:18.997833 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005dbe3e-1e18-417b-9ea8-ddd208089838 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:18.998296 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005dbe3e-1e18-417b-9ea8-ddd208089838 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104/groups/8d19401d46cc4ff697a7406a98eefcc6/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:19.006337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005dbe3e-1e18-417b-9ea8-ddd208089838 None admin] RBAC: Authorizing `identity:create_grant(project_id=96874534e5534d3699fb67c58215c104, group_id=8d19401d46cc4ff697a7406a98eefcc6, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:19.008022 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005dbe3e-1e18-417b-9ea8-ddd208089838 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:19.041369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 198/403] 10.222.0.60 () {58 vars in 1442 bytes} [Sat Sep 17 04:15:18 2022] PUT /identity/v3/projects/96874534e5534d3699fb67c58215c104/groups/8d19401d46cc4ff697a7406a98eefcc6/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:15:19.057941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26c6a7-c197-40b9-9fe7-b57623929bcd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:19.097713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64058f2-7b12-4f86-a17a-af545e855ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:19.098371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64058f2-7b12-4f86-a17a-af545e855ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:19.098651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64058f2-7b12-4f86-a17a-af545e855ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:19.098919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64058f2-7b12-4f86-a17a-af545e855ed8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:19.108749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:19.113735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64058f2-7b12-4f86-a17a-af545e855ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e64058f2-7b12-4f86-a17a-af545e855ed8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:19.119537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:19.120498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:19.121506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 206/404] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:15:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:19.128767 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005dbe3e-1e18-417b-9ea8-ddd208089838 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:19.283016 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005dbe3e-1e18-417b-9ea8-ddd208089838 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:19.319430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4af89-eea6-4588-8d88-c23b2eefaf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:19.320404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4af89-eea6-4588-8d88-c23b2eefaf85 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:19.320953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4af89-eea6-4588-8d88-c23b2eefaf85 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:19.321540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4af89-eea6-4588-8d88-c23b2eefaf85 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:19.322561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4af89-eea6-4588-8d88-c23b2eefaf85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:19.323051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4af89-eea6-4588-8d88-c23b2eefaf85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:19.324581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4af89-eea6-4588-8d88-c23b2eefaf85 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:19.333932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 199/405] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:19.341533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64058f2-7b12-4f86-a17a-af545e855ed8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:19.355357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bab07c-7cfd-4610-bd23-b10f186cba7f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:19.355948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bab07c-7cfd-4610-bd23-b10f186cba7f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:19.356469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bab07c-7cfd-4610-bd23-b10f186cba7f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:19.361071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 200/406] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:19.366585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d6e6a-cccf-4c93-bf83-2f60e9d8446c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:19.367534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d6e6a-cccf-4c93-bf83-2f60e9d8446c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:19.368009 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d6e6a-cccf-4c93-bf83-2f60e9d8446c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:19.382422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:19.388827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4b7d6e6a-cccf-4c93-bf83-2f60e9d8446c None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:19.390577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4b7d6e6a-cccf-4c93-bf83-2f60e9d8446c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:19.459514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 201/407] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:19.503199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64058f2-7b12-4f86-a17a-af545e855ed8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:19.558829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbaffb-a4aa-4cde-9e80-248cb690993c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:19.559379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbaffb-a4aa-4cde-9e80-248cb690993c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:19.559653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbaffb-a4aa-4cde-9e80-248cb690993c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:19.559922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbaffb-a4aa-4cde-9e80-248cb690993c None admin] PATH_INFO: `/v3/groups/8d19401d46cc4ff697a7406a98eefcc6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:19.565756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbaffb-a4aa-4cde-9e80-248cb690993c None admin] RBAC: Authorizing `identity:get_group(group_id=8d19401d46cc4ff697a7406a98eefcc6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:19.567202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbaffb-a4aa-4cde-9e80-248cb690993c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:19.568979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 207/408] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:19 2022] GET /identity/v3/groups/8d19401d46cc4ff697a7406a98eefcc6 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:19.574995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d6e6a-cccf-4c93-bf83-2f60e9d8446c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:19.768775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d6e6a-cccf-4c93-bf83-2f60e9d8446c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:19.827064 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbaffb-a4aa-4cde-9e80-248cb690993c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:19.839602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186214e4-c4a4-445f-b8a1-295337a8ba45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:19.840321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186214e4-c4a4-445f-b8a1-295337a8ba45 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:19.840597 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186214e4-c4a4-445f-b8a1-295337a8ba45 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:19.840879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186214e4-c4a4-445f-b8a1-295337a8ba45 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:19.842491 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186214e4-c4a4-445f-b8a1-295337a8ba45 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:19.844515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186214e4-c4a4-445f-b8a1-295337a8ba45 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:19.847994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 202/409] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:19 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:19.853234 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186214e4-c4a4-445f-b8a1-295337a8ba45 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:19.872536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:19.963085 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:20.020726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186214e4-c4a4-445f-b8a1-295337a8ba45 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:20.056953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61203af8-37fe-4cd5-a427-d64bb35b758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:20.057656 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61203af8-37fe-4cd5-a427-d64bb35b758b None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:20.057927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61203af8-37fe-4cd5-a427-d64bb35b758b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:20.058193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61203af8-37fe-4cd5-a427-d64bb35b758b None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104/groups/8d19401d46cc4ff697a7406a98eefcc6/roles/84212031716c4cf8a5083824fe47dbb7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:20.060326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61203af8-37fe-4cd5-a427-d64bb35b758b None admin] RBAC: Authorizing `identity:create_grant(project_id=96874534e5534d3699fb67c58215c104, group_id=8d19401d46cc4ff697a7406a98eefcc6, role_id=84212031716c4cf8a5083824fe47dbb7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:20.061788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61203af8-37fe-4cd5-a427-d64bb35b758b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:20.085663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 203/410] 10.222.0.60 () {58 vars in 1442 bytes} [Sat Sep 17 04:15:19 2022] PUT /identity/v3/projects/96874534e5534d3699fb67c58215c104/groups/8d19401d46cc4ff697a7406a98eefcc6/roles/84212031716c4cf8a5083824fe47dbb7 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:15:20.089217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbaffb-a4aa-4cde-9e80-248cb690993c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:20.113229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae20e35-faa7-4559-bdbd-d98b820c1eaa None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:20.113580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae20e35-faa7-4559-bdbd-d98b820c1eaa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:20.113812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae20e35-faa7-4559-bdbd-d98b820c1eaa None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:20.115843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 204/411] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:20.121780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af0f59-80ff-470f-ad68-52d05308bfce None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:20.122062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af0f59-80ff-470f-ad68-52d05308bfce None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:20.122331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af0f59-80ff-470f-ad68-52d05308bfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:20.149450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-33af0f59-80ff-470f-ad68-52d05308bfce None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:20.150728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-33af0f59-80ff-470f-ad68-52d05308bfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:20.176231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb2d81-8672-4322-a1b8-a6278a66cb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:20.177281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb2d81-8672-4322-a1b8-a6278a66cb37 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:20.177614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb2d81-8672-4322-a1b8-a6278a66cb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:20.177891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb2d81-8672-4322-a1b8-a6278a66cb37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:20.201470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb2d81-8672-4322-a1b8-a6278a66cb37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-5bdb2d81-8672-4322-a1b8-a6278a66cb37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:20.208332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:20.209311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:20.210127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:20.210127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:20.210127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:20.210127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:20.210127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:20.210127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:20.210127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:20.210127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:20.210127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:20.210127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:20.210127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:15:20.210127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:20.215907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 208/412] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:15:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:20.217224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb2d81-8672-4322-a1b8-a6278a66cb37 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:20.264926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:20.280806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 205/413] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:20.281677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a117d-3fd7-4b6b-be2b-0e30ddb25b9f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:20.281957 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a117d-3fd7-4b6b-be2b-0e30ddb25b9f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:20.282224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a117d-3fd7-4b6b-be2b-0e30ddb25b9f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:20.287535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 206/414] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:20.290344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3704c6-4ab3-41da-82f3-94a7ce416a8f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:20.290344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3704c6-4ab3-41da-82f3-94a7ce416a8f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:20.290344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3704c6-4ab3-41da-82f3-94a7ce416a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:20.318121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0f3704c6-4ab3-41da-82f3-94a7ce416a8f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:20.319298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0f3704c6-4ab3-41da-82f3-94a7ce416a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:20.374468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb2d81-8672-4322-a1b8-a6278a66cb37 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:20.396766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 207/415] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:20.402455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:20.410833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50f3fd-e079-462b-b042-daa26cf633e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:20.411494 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50f3fd-e079-462b-b042-daa26cf633e0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:20.411767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50f3fd-e079-462b-b042-daa26cf633e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:20.412035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50f3fd-e079-462b-b042-daa26cf633e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:20.412742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50f3fd-e079-462b-b042-daa26cf633e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:20.413043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50f3fd-e079-462b-b042-daa26cf633e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:20.414498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50f3fd-e079-462b-b042-daa26cf633e0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:20.425160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 209/416] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:20.430856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3704c6-4ab3-41da-82f3-94a7ce416a8f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:20.585036 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3704c6-4ab3-41da-82f3-94a7ce416a8f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:20.614744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:20.636744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50f3fd-e079-462b-b042-daa26cf633e0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:20.639134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5c797-2489-4b4a-884d-54eef2bc8e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:20.639906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5c797-2489-4b4a-884d-54eef2bc8e9b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:20.640269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5c797-2489-4b4a-884d-54eef2bc8e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:20.640540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5c797-2489-4b4a-884d-54eef2bc8e9b None admin] PATH_INFO: `/v3/groups/13cad0fff80a4dfdba252827031bb0f8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:20.641798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5c797-2489-4b4a-884d-54eef2bc8e9b None admin] RBAC: Authorizing `identity:get_group(group_id=13cad0fff80a4dfdba252827031bb0f8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:20.643239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5c797-2489-4b4a-884d-54eef2bc8e9b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:20.644953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 208/417] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:20 2022] GET /identity/v3/groups/13cad0fff80a4dfdba252827031bb0f8 => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:20.652464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5c797-2489-4b4a-884d-54eef2bc8e9b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:20.773011 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5c797-2489-4b4a-884d-54eef2bc8e9b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:20.790009 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50f3fd-e079-462b-b042-daa26cf633e0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:20.814668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319d357-050f-4461-97d1-68d99354081a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:20.815936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319d357-050f-4461-97d1-68d99354081a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:20.816224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319d357-050f-4461-97d1-68d99354081a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:20.816633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319d357-050f-4461-97d1-68d99354081a None admin] PATH_INFO: `/v3/projects/7fd2fb11102f467d836ef30d6efdd922` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:20.818173 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319d357-050f-4461-97d1-68d99354081a None admin] RBAC: Authorizing `identity:get_project(project_id=7fd2fb11102f467d836ef30d6efdd922)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:20.820322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319d357-050f-4461-97d1-68d99354081a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:20.823263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 209/418] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:20 2022] GET /identity/v3/projects/7fd2fb11102f467d836ef30d6efdd922 => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:20.826698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c1823-5b83-4765-854a-a352d9388788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:20.827564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c1823-5b83-4765-854a-a352d9388788 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:20.828030 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c1823-5b83-4765-854a-a352d9388788 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:20.828545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c1823-5b83-4765-854a-a352d9388788 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:20.828803 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319d357-050f-4461-97d1-68d99354081a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:20.844903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c1823-5b83-4765-854a-a352d9388788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-0e6c1823-5b83-4765-854a-a352d9388788 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:20.851236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:20.852125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:20.852972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 210/419] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:20.857560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c1823-5b83-4765-854a-a352d9388788 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:20.897422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:20.929836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:20.941720 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319d357-050f-4461-97d1-68d99354081a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:20.978650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa4979-1042-4e28-9e58-76cf4d54c998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:20.979747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa4979-1042-4e28-9e58-76cf4d54c998 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:20.980215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa4979-1042-4e28-9e58-76cf4d54c998 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:20.980679 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa4979-1042-4e28-9e58-76cf4d54c998 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:20.992234 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa4979-1042-4e28-9e58-76cf4d54c998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:20.995216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c1823-5b83-4765-854a-a352d9388788 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a2aa4979-1042-4e28-9e58-76cf4d54c998 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:20.997729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:20.998589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:20.999414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:20.999414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:20.999414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:20.999414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:20.999414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:20.999414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:20.999414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:20.999414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:20.999414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:20.999414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:20.999414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:20.999414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:21.002174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 210/420] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:15:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:21.003182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa4979-1042-4e28-9e58-76cf4d54c998 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:21.040870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f56f4-0ba3-41f7-9320-5872a4b40ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:21.041610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f56f4-0ba3-41f7-9320-5872a4b40ea8 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:21.041886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f56f4-0ba3-41f7-9320-5872a4b40ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:21.042154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f56f4-0ba3-41f7-9320-5872a4b40ea8 None admin] PATH_INFO: `/v3/projects/7fd2fb11102f467d836ef30d6efdd922/groups/13cad0fff80a4dfdba252827031bb0f8/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:21.045860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f56f4-0ba3-41f7-9320-5872a4b40ea8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fd2fb11102f467d836ef30d6efdd922, group_id=13cad0fff80a4dfdba252827031bb0f8, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:21.047164 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f56f4-0ba3-41f7-9320-5872a4b40ea8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:21.061699 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 211/421] 10.222.0.60 () {58 vars in 1442 bytes} [Sat Sep 17 04:15:20 2022] PUT /identity/v3/projects/7fd2fb11102f467d836ef30d6efdd922/groups/13cad0fff80a4dfdba252827031bb0f8/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:15:21.062617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f56f4-0ba3-41f7-9320-5872a4b40ea8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:21.123561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa4979-1042-4e28-9e58-76cf4d54c998 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:21.148475 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:21.165578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450806e-4677-4d97-96db-775f647fff4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:21.166623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450806e-4677-4d97-96db-775f647fff4e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:21.167143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450806e-4677-4d97-96db-775f647fff4e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:21.167631 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450806e-4677-4d97-96db-775f647fff4e None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:21.168495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450806e-4677-4d97-96db-775f647fff4e None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:21.169107 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450806e-4677-4d97-96db-775f647fff4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:21.170789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450806e-4677-4d97-96db-775f647fff4e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:21.175931 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f56f4-0ba3-41f7-9320-5872a4b40ea8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:21.181350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 211/422] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:21.192799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450806e-4677-4d97-96db-775f647fff4e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:21.215539 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:21.235612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d733376-5082-4aff-a935-0821a337a8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:21.236247 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d733376-5082-4aff-a935-0821a337a8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:21.236518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d733376-5082-4aff-a935-0821a337a8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:21.239560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d733376-5082-4aff-a935-0821a337a8a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:21.242663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d733376-5082-4aff-a935-0821a337a8a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:21.242663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d733376-5082-4aff-a935-0821a337a8a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:21.245495 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d733376-5082-4aff-a935-0821a337a8a4 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:21.265405 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 212/423] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:21.270380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d733376-5082-4aff-a935-0821a337a8a4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:21.395454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450806e-4677-4d97-96db-775f647fff4e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:21.431684 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d733376-5082-4aff-a935-0821a337a8a4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:21.434493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234edbab-53f5-4324-aebb-a109ae4fac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:21.435162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234edbab-53f5-4324-aebb-a109ae4fac5c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:21.435433 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234edbab-53f5-4324-aebb-a109ae4fac5c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:21.435700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234edbab-53f5-4324-aebb-a109ae4fac5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:21.471623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38ebf0-0c4b-48d3-a462-efd148440782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:21.473104 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38ebf0-0c4b-48d3-a462-efd148440782 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:21.473389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38ebf0-0c4b-48d3-a462-efd148440782 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:21.473664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38ebf0-0c4b-48d3-a462-efd148440782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:21.502534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:21.527248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:21.545913 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234edbab-53f5-4324-aebb-a109ae4fac5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:21.547410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234edbab-53f5-4324-aebb-a109ae4fac5c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:21.557854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38ebf0-0c4b-48d3-a462-efd148440782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:21.561546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38ebf0-0c4b-48d3-a462-efd148440782 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:21.675538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 213/424] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:21.683117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38ebf0-0c4b-48d3-a462-efd148440782 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:21.763423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:21.765417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 212/425] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 04:15:21.767380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c4de1-9487-40b4-a332-485f965fb220 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:21.767862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c4de1-9487-40b4-a332-485f965fb220 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:21.768793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c4de1-9487-40b4-a332-485f965fb220 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:21.771367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 213/426] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:21.776084 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c4de1-9487-40b4-a332-485f965fb220 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:21.807529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38ebf0-0c4b-48d3-a462-efd148440782 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:21.848361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e1e19-ef7a-4f2f-b5e5-3a031a00ce5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:21.849381 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e1e19-ef7a-4f2f-b5e5-3a031a00ce5b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:21.849381 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e1e19-ef7a-4f2f-b5e5-3a031a00ce5b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:21.849621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e1e19-ef7a-4f2f-b5e5-3a031a00ce5b None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:21.850632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e1e19-ef7a-4f2f-b5e5-3a031a00ce5b None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:21.852061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e1e19-ef7a-4f2f-b5e5-3a031a00ce5b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:21.854577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 214/427] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:21 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:21.859363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e69418-406d-4eb3-b29c-391515fc271a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:21.859643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e69418-406d-4eb3-b29c-391515fc271a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:21.859917 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e69418-406d-4eb3-b29c-391515fc271a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:21.868786 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:21.879331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c5e69418-406d-4eb3-b29c-391515fc271a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:21.880519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c5e69418-406d-4eb3-b29c-391515fc271a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:21.893925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c4de1-9487-40b4-a332-485f965fb220 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:21.940215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ff0b8-1382-46e8-8619-f27bbdcd41c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:21.941369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ff0b8-1382-46e8-8619-f27bbdcd41c3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:21.941948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ff0b8-1382-46e8-8619-f27bbdcd41c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:21.942436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ff0b8-1382-46e8-8619-f27bbdcd41c3 None admin] PATH_INFO: `/v3/projects/d7203986da504ab1ba9f76d7f64f7dce` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:21.944030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ff0b8-1382-46e8-8619-f27bbdcd41c3 None admin] RBAC: Authorizing `identity:get_project(project_id=d7203986da504ab1ba9f76d7f64f7dce)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:21.946370 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ff0b8-1382-46e8-8619-f27bbdcd41c3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:21.949428 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 214/428] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:21 2022] GET /identity/v3/projects/d7203986da504ab1ba9f76d7f64f7dce => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:21.950682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ff0b8-1382-46e8-8619-f27bbdcd41c3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:21.957131 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 215/429] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:21.960677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e69418-406d-4eb3-b29c-391515fc271a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:22.060630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ff0b8-1382-46e8-8619-f27bbdcd41c3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:22.099378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:22.112818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556da3a-d7ad-4ef7-a023-4d73688d9430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:22.113496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556da3a-d7ad-4ef7-a023-4d73688d9430 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:22.113766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556da3a-d7ad-4ef7-a023-4d73688d9430 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:22.114027 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556da3a-d7ad-4ef7-a023-4d73688d9430 None admin] PATH_INFO: `/v3/groups/8d19401d46cc4ff697a7406a98eefcc6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:22.115925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556da3a-d7ad-4ef7-a023-4d73688d9430 None admin] RBAC: Authorizing `identity:get_group(group_id=8d19401d46cc4ff697a7406a98eefcc6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:22.117971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556da3a-d7ad-4ef7-a023-4d73688d9430 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:22.119614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 215/430] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:21 2022] GET /identity/v3/groups/8d19401d46cc4ff697a7406a98eefcc6 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:22.125648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556da3a-d7ad-4ef7-a023-4d73688d9430 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:22.150675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e69418-406d-4eb3-b29c-391515fc271a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:22.164116 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:22.211449 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962a11f-f5bd-486b-be99-10654253467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:22.212078 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962a11f-f5bd-486b-be99-10654253467b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:22.212347 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962a11f-f5bd-486b-be99-10654253467b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:22.212611 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962a11f-f5bd-486b-be99-10654253467b None admin] PATH_INFO: `/v3/groups/8d19401d46cc4ff697a7406a98eefcc6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:22.215722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962a11f-f5bd-486b-be99-10654253467b None admin] RBAC: Authorizing `identity:get_group(group_id=8d19401d46cc4ff697a7406a98eefcc6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:22.217173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962a11f-f5bd-486b-be99-10654253467b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:22.219051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 216/431] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:21 2022] GET /identity/v3/groups/8d19401d46cc4ff697a7406a98eefcc6 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:22.224823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962a11f-f5bd-486b-be99-10654253467b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:22.286643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556da3a-d7ad-4ef7-a023-4d73688d9430 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:22.322609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b1ee6-d41f-499d-ab4e-74022c304ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:22.323243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b1ee6-d41f-499d-ab4e-74022c304ceb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:22.323513 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b1ee6-d41f-499d-ab4e-74022c304ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:22.323872 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b1ee6-d41f-499d-ab4e-74022c304ceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:22.324600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b1ee6-d41f-499d-ab4e-74022c304ceb None admin] RBAC: Adding query filter params (group.id=8d19401d46cc4ff697a7406a98eefcc6, role.id=84212031716c4cf8a5083824fe47dbb7, scope.project.id=d7203986da504ab1ba9f76d7f64f7dce) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:22.324898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b1ee6-d41f-499d-ab4e-74022c304ceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:22.326443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b1ee6-d41f-499d-ab4e-74022c304ceb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:22.337185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 216/432] 10.222.0.60 () {56 vars in 1376 bytes} [Sat Sep 17 04:15:22 2022] GET /identity/v3/role_assignments?group.id=8d19401d46cc4ff697a7406a98eefcc6&scope.project.id=d7203986da504ab1ba9f76d7f64f7dce&role.id=84212031716c4cf8a5083824fe47dbb7 => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:22.338112 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b1ee6-d41f-499d-ab4e-74022c304ceb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:22.366581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962a11f-f5bd-486b-be99-10654253467b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:22.444572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af4d7e-aaaa-4553-9a32-57777fc5994f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:22.445624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af4d7e-aaaa-4553-9a32-57777fc5994f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:22.446122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af4d7e-aaaa-4553-9a32-57777fc5994f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:22.446722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af4d7e-aaaa-4553-9a32-57777fc5994f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:22.447711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af4d7e-aaaa-4553-9a32-57777fc5994f None admin] RBAC: Adding query filter params (group.id=8d19401d46cc4ff697a7406a98eefcc6, role.id=231a681c35614ea3aaf3452a8c167c5c, scope.project.id=d7203986da504ab1ba9f76d7f64f7dce) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:22.448906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af4d7e-aaaa-4553-9a32-57777fc5994f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:22.451312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af4d7e-aaaa-4553-9a32-57777fc5994f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:22.466248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 217/433] 10.222.0.60 () {56 vars in 1376 bytes} [Sat Sep 17 04:15:22 2022] GET /identity/v3/role_assignments?group.id=8d19401d46cc4ff697a7406a98eefcc6&scope.project.id=d7203986da504ab1ba9f76d7f64f7dce&role.id=231a681c35614ea3aaf3452a8c167c5c => generated 611 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:22.477830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:22.513825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b1ee6-d41f-499d-ab4e-74022c304ceb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:22.561179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269bb1c-500d-4f91-a7a6-ba8d87ef24a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:22.561829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269bb1c-500d-4f91-a7a6-ba8d87ef24a9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:22.562107 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269bb1c-500d-4f91-a7a6-ba8d87ef24a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:22.562369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269bb1c-500d-4f91-a7a6-ba8d87ef24a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:22.574794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269bb1c-500d-4f91-a7a6-ba8d87ef24a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4269bb1c-500d-4f91-a7a6-ba8d87ef24a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:22.580558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:22.581550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:22.582441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 217/434] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:22.590290 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af4d7e-aaaa-4553-9a32-57777fc5994f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:22.594025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:22.618981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72588d-01fb-4ce4-af2b-6e1b7e724ce5 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:22.619327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72588d-01fb-4ce4-af2b-6e1b7e724ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:22.619517 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72588d-01fb-4ce4-af2b-6e1b7e724ce5 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:22.621033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 218/435] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:22.627009 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e36394-0ee1-4e50-a244-4aac5cccfa94 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:22.627708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e36394-0ee1-4e50-a244-4aac5cccfa94 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:22.628165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e36394-0ee1-4e50-a244-4aac5cccfa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:22.650772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d3e36394-0ee1-4e50-a244-4aac5cccfa94 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:22.653238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d3e36394-0ee1-4e50-a244-4aac5cccfa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:22.733598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af4d7e-aaaa-4553-9a32-57777fc5994f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:22.739227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 219/436] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:22.768612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20824c82-5989-450c-8dac-dd3c3508b209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:22.769534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20824c82-5989-450c-8dac-dd3c3508b209 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:22.769945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20824c82-5989-450c-8dac-dd3c3508b209 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:22.770369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20824c82-5989-450c-8dac-dd3c3508b209 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:22.771181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20824c82-5989-450c-8dac-dd3c3508b209 None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:22.771613 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20824c82-5989-450c-8dac-dd3c3508b209 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:22.773131 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20824c82-5989-450c-8dac-dd3c3508b209 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:22.782772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 218/437] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:15:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:22.789104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e36394-0ee1-4e50-a244-4aac5cccfa94 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:22.861530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:22.892357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e36394-0ee1-4e50-a244-4aac5cccfa94 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:22.931066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180157ed-3634-4f7d-829e-efa162553b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:22.931754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180157ed-3634-4f7d-829e-efa162553b58 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:22.932043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180157ed-3634-4f7d-829e-efa162553b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:22.932335 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180157ed-3634-4f7d-829e-efa162553b58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:23.002702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180157ed-3634-4f7d-829e-efa162553b58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:23.004158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180157ed-3634-4f7d-829e-efa162553b58 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:23.011305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20824c82-5989-450c-8dac-dd3c3508b209 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:23.085347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:23.089231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:23.125707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 220/438] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:23.131846 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180157ed-3634-4f7d-829e-efa162553b58 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:23.139978 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20824c82-5989-450c-8dac-dd3c3508b209 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:23.192117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca7380-3a14-4991-8aa1-fcbd6fa004df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:23.192775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca7380-3a14-4991-8aa1-fcbd6fa004df None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:23.193054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca7380-3a14-4991-8aa1-fcbd6fa004df None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:23.193386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca7380-3a14-4991-8aa1-fcbd6fa004df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:23.206024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca7380-3a14-4991-8aa1-fcbd6fa004df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c5ca7380-3a14-4991-8aa1-fcbd6fa004df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:23.211142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:23.212086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:23.213338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 219/439] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:15:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:23.218902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca7380-3a14-4991-8aa1-fcbd6fa004df None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:23.277399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180157ed-3634-4f7d-829e-efa162553b58 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:23.320136 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e447d3f-9d51-4a1d-8278-6cd312835961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:23.320784 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e447d3f-9d51-4a1d-8278-6cd312835961 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:23.321065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e447d3f-9d51-4a1d-8278-6cd312835961 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:23.321424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e447d3f-9d51-4a1d-8278-6cd312835961 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:23.322566 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e447d3f-9d51-4a1d-8278-6cd312835961 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:23.324010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e447d3f-9d51-4a1d-8278-6cd312835961 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:23.325838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 221/440] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:23 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:23.331015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e447d3f-9d51-4a1d-8278-6cd312835961 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:23.337672 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca7380-3a14-4991-8aa1-fcbd6fa004df None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:23.361647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:23.373608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:23.378024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9af283-502a-4b2f-b653-20636e50c12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:23.378630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9af283-502a-4b2f-b653-20636e50c12f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:23.379108 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9af283-502a-4b2f-b653-20636e50c12f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:23.379221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9af283-502a-4b2f-b653-20636e50c12f None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:23.379854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9af283-502a-4b2f-b653-20636e50c12f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:23.380105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9af283-502a-4b2f-b653-20636e50c12f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:23.381469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9af283-502a-4b2f-b653-20636e50c12f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:23.390023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 220/441] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:23.395581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9af283-502a-4b2f-b653-20636e50c12f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:23.432882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e447d3f-9d51-4a1d-8278-6cd312835961 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:23.471219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15107397-b710-4c4b-b2bb-5d82a0f3f98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:23.471853 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15107397-b710-4c4b-b2bb-5d82a0f3f98c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:23.472120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15107397-b710-4c4b-b2bb-5d82a0f3f98c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:23.472384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15107397-b710-4c4b-b2bb-5d82a0f3f98c None admin] PATH_INFO: `/v3/groups/8d19401d46cc4ff697a7406a98eefcc6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:23.473516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15107397-b710-4c4b-b2bb-5d82a0f3f98c None admin] RBAC: Authorizing `identity:get_group(group_id=8d19401d46cc4ff697a7406a98eefcc6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:23.475284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15107397-b710-4c4b-b2bb-5d82a0f3f98c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:23.476722 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 222/442] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:23 2022] GET /identity/v3/groups/8d19401d46cc4ff697a7406a98eefcc6 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:23.482238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15107397-b710-4c4b-b2bb-5d82a0f3f98c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:23.502338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9af283-502a-4b2f-b653-20636e50c12f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:23.538002 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92922e56-16e6-4f3e-a348-3b50c737aac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:23.538628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92922e56-16e6-4f3e-a348-3b50c737aac8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:23.538906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92922e56-16e6-4f3e-a348-3b50c737aac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:23.539177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92922e56-16e6-4f3e-a348-3b50c737aac8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:23.579062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15107397-b710-4c4b-b2bb-5d82a0f3f98c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:23.588166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:23.593508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:23.608054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92922e56-16e6-4f3e-a348-3b50c737aac8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:23.609552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92922e56-16e6-4f3e-a348-3b50c737aac8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:23.615346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b8ac8f-3d01-4e0f-b6f8-2b5b9740338b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:23.615951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8ac8f-3d01-4e0f-b6f8-2b5b9740338b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:23.616222 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8ac8f-3d01-4e0f-b6f8-2b5b9740338b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:23.616492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8ac8f-3d01-4e0f-b6f8-2b5b9740338b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:23.617642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b8ac8f-3d01-4e0f-b6f8-2b5b9740338b None admin] RBAC: Adding query filter params (group.id=8d19401d46cc4ff697a7406a98eefcc6, role.id=231a681c35614ea3aaf3452a8c167c5c, scope.project.id=96874534e5534d3699fb67c58215c104) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:23.617944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b8ac8f-3d01-4e0f-b6f8-2b5b9740338b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:23.619460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b8ac8f-3d01-4e0f-b6f8-2b5b9740338b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:23.625497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 223/443] 10.222.0.60 () {56 vars in 1376 bytes} [Sat Sep 17 04:15:23 2022] GET /identity/v3/role_assignments?group.id=8d19401d46cc4ff697a7406a98eefcc6&scope.project.id=96874534e5534d3699fb67c58215c104&role.id=231a681c35614ea3aaf3452a8c167c5c => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:23.626598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836195d-c5cb-4eeb-a3c4-3828817dff92 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:23.626880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836195d-c5cb-4eeb-a3c4-3828817dff92 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:23.627148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836195d-c5cb-4eeb-a3c4-3828817dff92 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:23.628347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 224/444] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:23.634488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace31f07-7420-4d70-bfa6-2d012489baa7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:23.634749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace31f07-7420-4d70-bfa6-2d012489baa7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:23.635007 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace31f07-7420-4d70-bfa6-2d012489baa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:23.656021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ace31f07-7420-4d70-bfa6-2d012489baa7 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:23.657281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ace31f07-7420-4d70-bfa6-2d012489baa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:23.729685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 221/445] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 04:15:23.735589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92922e56-16e6-4f3e-a348-3b50c737aac8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:23.743105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 225/446] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:23.825180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:23.840328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92922e56-16e6-4f3e-a348-3b50c737aac8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:23.875280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf26123-4607-4d54-939d-435ac26faad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:23.875919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf26123-4607-4d54-939d-435ac26faad9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:23.876194 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf26123-4607-4d54-939d-435ac26faad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:23.876467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf26123-4607-4d54-939d-435ac26faad9 None admin] PATH_INFO: `/v3/projects/96874534e5534d3699fb67c58215c104` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:23.877631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf26123-4607-4d54-939d-435ac26faad9 None admin] RBAC: Authorizing `identity:get_project(project_id=96874534e5534d3699fb67c58215c104)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:23.879066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf26123-4607-4d54-939d-435ac26faad9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:23.880869 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 222/447] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:23 2022] GET /identity/v3/projects/96874534e5534d3699fb67c58215c104 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:23.886617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace31f07-7420-4d70-bfa6-2d012489baa7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:23.991152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace31f07-7420-4d70-bfa6-2d012489baa7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:23.993482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:24.010607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf26123-4607-4d54-939d-435ac26faad9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:24.027295 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405be846-52fc-4cc9-8380-2208f21e6dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:24.028273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405be846-52fc-4cc9-8380-2208f21e6dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:24.028273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405be846-52fc-4cc9-8380-2208f21e6dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:24.028500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405be846-52fc-4cc9-8380-2208f21e6dd8 None admin] PATH_INFO: `/v3/groups/8d19401d46cc4ff697a7406a98eefcc6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:24.030212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405be846-52fc-4cc9-8380-2208f21e6dd8 None admin] RBAC: Authorizing `identity:get_group(group_id=8d19401d46cc4ff697a7406a98eefcc6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:24.031664 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405be846-52fc-4cc9-8380-2208f21e6dd8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:24.033285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 226/448] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:23 2022] GET /identity/v3/groups/8d19401d46cc4ff697a7406a98eefcc6 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:24.039116 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405be846-52fc-4cc9-8380-2208f21e6dd8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:24.117935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf26123-4607-4d54-939d-435ac26faad9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:24.134249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405be846-52fc-4cc9-8380-2208f21e6dd8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:24.150916 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8db5ff-e7b7-4b3a-98b3-5e75185910ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:24.151547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8db5ff-e7b7-4b3a-98b3-5e75185910ab None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:24.151813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8db5ff-e7b7-4b3a-98b3-5e75185910ab None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:24.152080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8db5ff-e7b7-4b3a-98b3-5e75185910ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:24.162155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8db5ff-e7b7-4b3a-98b3-5e75185910ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:24.166881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f8cb0-3d68-4a70-acb1-03a29744cd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-4a8db5ff-e7b7-4b3a-98b3-5e75185910ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:24.167004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:24.168021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:24.168922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:24.168922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:24.168922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:24.168922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:24.168922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:24.168922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:24.168922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:24.168922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:24.168922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:24.168922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:24.168922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:15:24.168922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:24.169763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f8cb0-3d68-4a70-acb1-03a29744cd59 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:24.169763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f8cb0-3d68-4a70-acb1-03a29744cd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:24.169763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f8cb0-3d68-4a70-acb1-03a29744cd59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:24.169942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 223/449] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:15:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:24.170005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f8cb0-3d68-4a70-acb1-03a29744cd59 None admin] RBAC: Adding query filter params (group.id=8d19401d46cc4ff697a7406a98eefcc6, role.id=84212031716c4cf8a5083824fe47dbb7, scope.project.id=96874534e5534d3699fb67c58215c104) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:24.170005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f8cb0-3d68-4a70-acb1-03a29744cd59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:24.170416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f8cb0-3d68-4a70-acb1-03a29744cd59 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:24.173896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8db5ff-e7b7-4b3a-98b3-5e75185910ab None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:24.177112 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 227/450] 10.222.0.60 () {56 vars in 1376 bytes} [Sat Sep 17 04:15:24 2022] GET /identity/v3/role_assignments?group.id=8d19401d46cc4ff697a7406a98eefcc6&scope.project.id=96874534e5534d3699fb67c58215c104&role.id=84212031716c4cf8a5083824fe47dbb7 => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:24.190222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:24.288197 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8db5ff-e7b7-4b3a-98b3-5e75185910ab None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:24.326430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94e35e-7058-4c50-8fcc-b74e066f36e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:24.327049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94e35e-7058-4c50-8fcc-b74e066f36e7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:24.327329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94e35e-7058-4c50-8fcc-b74e066f36e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:24.327608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94e35e-7058-4c50-8fcc-b74e066f36e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:24.328282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94e35e-7058-4c50-8fcc-b74e066f36e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:24.328583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94e35e-7058-4c50-8fcc-b74e066f36e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:24.330029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94e35e-7058-4c50-8fcc-b74e066f36e7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:24.339403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 224/451] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:24.345512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f8cb0-3d68-4a70-acb1-03a29744cd59 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:24.384339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:24.451226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f8cb0-3d68-4a70-acb1-03a29744cd59 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:24.485132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd4fb1-8afd-40c7-9e28-ee5e07c3131f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:24.485756 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd4fb1-8afd-40c7-9e28-ee5e07c3131f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:24.486020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd4fb1-8afd-40c7-9e28-ee5e07c3131f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:24.486287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd4fb1-8afd-40c7-9e28-ee5e07c3131f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:24.554077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd4fb1-8afd-40c7-9e28-ee5e07c3131f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:24.555540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd4fb1-8afd-40c7-9e28-ee5e07c3131f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:24.589505 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:24.658732 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 228/452] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:24.664944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94e35e-7058-4c50-8fcc-b74e066f36e7 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:24.760647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:24.784535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94e35e-7058-4c50-8fcc-b74e066f36e7 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:24.818644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745b31f-61f2-4260-861f-330d8c632541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:24.819272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745b31f-61f2-4260-861f-330d8c632541 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:24.819532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745b31f-61f2-4260-861f-330d8c632541 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:24.819824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745b31f-61f2-4260-861f-330d8c632541 None admin] PATH_INFO: `/v3/projects/7fd2fb11102f467d836ef30d6efdd922` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:24.821107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745b31f-61f2-4260-861f-330d8c632541 None admin] RBAC: Authorizing `identity:get_project(project_id=7fd2fb11102f467d836ef30d6efdd922)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:24.822620 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745b31f-61f2-4260-861f-330d8c632541 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:24.824246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 225/453] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:15:24 2022] GET /identity/v3/projects/7fd2fb11102f467d836ef30d6efdd922 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:24.833137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd4fb1-8afd-40c7-9e28-ee5e07c3131f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:24.938726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd4fb1-8afd-40c7-9e28-ee5e07c3131f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:24.974057 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1f40a-87c1-4764-9e07-0c2c5d81fd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:24.975104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1f40a-87c1-4764-9e07-0c2c5d81fd01 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:24.975387 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1f40a-87c1-4764-9e07-0c2c5d81fd01 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:24.975658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1f40a-87c1-4764-9e07-0c2c5d81fd01 None admin] PATH_INFO: `/v3/groups/13cad0fff80a4dfdba252827031bb0f8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:24.976884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1f40a-87c1-4764-9e07-0c2c5d81fd01 None admin] RBAC: Authorizing `identity:get_group(group_id=13cad0fff80a4dfdba252827031bb0f8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:24.978398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1f40a-87c1-4764-9e07-0c2c5d81fd01 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:24.979936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 229/454] 10.222.0.60 () {56 vars in 1180 bytes} [Sat Sep 17 04:15:24 2022] GET /identity/v3/groups/13cad0fff80a4dfdba252827031bb0f8 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:24.986536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745b31f-61f2-4260-861f-330d8c632541 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:25.103835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745b31f-61f2-4260-861f-330d8c632541 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:25.138716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde89cd6-14f0-4c1e-8d76-404cda38fdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:25.139303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde89cd6-14f0-4c1e-8d76-404cda38fdd8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:25.139579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde89cd6-14f0-4c1e-8d76-404cda38fdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:25.139853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde89cd6-14f0-4c1e-8d76-404cda38fdd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:25.140579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde89cd6-14f0-4c1e-8d76-404cda38fdd8 None admin] RBAC: Adding query filter params (group.id=13cad0fff80a4dfdba252827031bb0f8, role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=7fd2fb11102f467d836ef30d6efdd922) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:25.140887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde89cd6-14f0-4c1e-8d76-404cda38fdd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:25.142355 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde89cd6-14f0-4c1e-8d76-404cda38fdd8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:25.148136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 226/455] 10.222.0.60 () {56 vars in 1376 bytes} [Sat Sep 17 04:15:24 2022] GET /identity/v3/role_assignments?group.id=13cad0fff80a4dfdba252827031bb0f8&scope.project.id=7fd2fb11102f467d836ef30d6efdd922&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:27.857875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc21c60-abb2-485f-96aa-ead41b95eebc None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:27.858441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc21c60-abb2-485f-96aa-ead41b95eebc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:27.858441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc21c60-abb2-485f-96aa-ead41b95eebc None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:27.860040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 230/456] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:27.865697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90875ca3-f175-48d4-9cda-d56023f66050 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:27.866608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90875ca3-f175-48d4-9cda-d56023f66050 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:27.867470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90875ca3-f175-48d4-9cda-d56023f66050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:27.890583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-90875ca3-f175-48d4-9cda-d56023f66050 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:27.892128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-90875ca3-f175-48d4-9cda-d56023f66050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:27.894468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:27.904819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f81ffb-4eb3-47d7-a22a-acdc845eaf9b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:27.905369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f81ffb-4eb3-47d7-a22a-acdc845eaf9b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:27.905876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f81ffb-4eb3-47d7-a22a-acdc845eaf9b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:27.907652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 231/457] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:27.912512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93b5e5-de3a-48aa-a42b-413e163e0aaa None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:27.912999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93b5e5-de3a-48aa-a42b-413e163e0aaa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:27.913477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93b5e5-de3a-48aa-a42b-413e163e0aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:27.924483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:27.952102 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-af93b5e5-de3a-48aa-a42b-413e163e0aaa None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:27.953329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-af93b5e5-de3a-48aa-a42b-413e163e0aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:28.002436 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 227/458] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:28.068578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 232/459] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:28.088223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d0a47-513d-4229-b443-842c6e29a601 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:28.088503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d0a47-513d-4229-b443-842c6e29a601 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:28.088928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d0a47-513d-4229-b443-842c6e29a601 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:28.090718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 228/460] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:28.096048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e607bca-1bd9-4c51-bd57-8a607cbd9a72 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:28.096337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e607bca-1bd9-4c51-bd57-8a607cbd9a72 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:28.096777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e607bca-1bd9-4c51-bd57-8a607cbd9a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:28.123034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8e607bca-1bd9-4c51-bd57-8a607cbd9a72 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:28.124440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8e607bca-1bd9-4c51-bd57-8a607cbd9a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:28.219342 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 233/461] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:28.378112 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d0a47-513d-4229-b443-842c6e29a601 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:28.410910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e607bca-1bd9-4c51-bd57-8a607cbd9a72 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:28.501540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:28.537590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d0a47-513d-4229-b443-842c6e29a601 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:28.548762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e607bca-1bd9-4c51-bd57-8a607cbd9a72 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:28.573023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea143a64-850b-45b9-8c9c-87b8268af86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:28.573682 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea143a64-850b-45b9-8c9c-87b8268af86a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:28.573968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea143a64-850b-45b9-8c9c-87b8268af86a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:28.574216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea143a64-850b-45b9-8c9c-87b8268af86a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:28.580658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:28.584789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea143a64-850b-45b9-8c9c-87b8268af86a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:28.586771 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea143a64-850b-45b9-8c9c-87b8268af86a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:28.588560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf02d0-260b-48c6-8534-797545bcf0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:28.589517 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf02d0-260b-48c6-8534-797545bcf0ad None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:28.589798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf02d0-260b-48c6-8534-797545bcf0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:28.590067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf02d0-260b-48c6-8534-797545bcf0ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:28.601259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf02d0-260b-48c6-8534-797545bcf0ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:28.602692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf02d0-260b-48c6-8534-797545bcf0ad None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-ea143a64-850b-45b9-8c9c-87b8268af86a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:28.602925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:28.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:28.604718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:28.604718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:28.604718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:28.604891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 229/462] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:28.606229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d327480-7b24-4434-ae4b-329922b346b1 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:28.606507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d327480-7b24-4434-ae4b-329922b346b1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:28.606779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d327480-7b24-4434-ae4b-329922b346b1 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:28.608005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 230/463] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:28.608693 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d327480-7b24-4434-ae4b-329922b346b1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-22bf02d0-260b-48c6-8534-797545bcf0ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:28.622633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:28.623682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:28.624537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:28.624537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:28.624537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:28.624537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 234/464] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:28.624537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf02d0-260b-48c6-8534-797545bcf0ad None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:28.727629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d327480-7b24-4434-ae4b-329922b346b1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:28.729174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf02d0-260b-48c6-8534-797545bcf0ad None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:28.744866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:28.762763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50aee6-2f66-4023-b421-98df7ac8c318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:28.763324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50aee6-2f66-4023-b421-98df7ac8c318 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:28.763599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50aee6-2f66-4023-b421-98df7ac8c318 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:28.763866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50aee6-2f66-4023-b421-98df7ac8c318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:28.765918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864e6b5-5b53-486e-90ae-ac70a945069e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:28.766531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864e6b5-5b53-486e-90ae-ac70a945069e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:28.766808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864e6b5-5b53-486e-90ae-ac70a945069e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:28.767077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864e6b5-5b53-486e-90ae-ac70a945069e None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:28.767719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864e6b5-5b53-486e-90ae-ac70a945069e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:28.768016 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864e6b5-5b53-486e-90ae-ac70a945069e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:28.769416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864e6b5-5b53-486e-90ae-ac70a945069e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:28.776584 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50aee6-2f66-4023-b421-98df7ac8c318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:28.778066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50aee6-2f66-4023-b421-98df7ac8c318 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:28.781977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 235/465] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:28.783371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a87559-b283-4d55-9813-b8d65482c2c1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:28.783804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a87559-b283-4d55-9813-b8d65482c2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:28.784256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a87559-b283-4d55-9813-b8d65482c2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-3d50aee6-2f66-4023-b421-98df7ac8c318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:28.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:28.796328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:28.797306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:28.797306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:28.797306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:28.797306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 231/466] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:28.801971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50aee6-2f66-4023-b421-98df7ac8c318 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:28.804177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-98a87559-b283-4d55-9813-b8d65482c2c1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:28.805598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-98a87559-b283-4d55-9813-b8d65482c2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:28.845786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:28.885575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 236/467] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:28.885575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a87559-b283-4d55-9813-b8d65482c2c1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:28.917736 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50aee6-2f66-4023-b421-98df7ac8c318 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:28.958268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd39ec-f978-4d73-a4e9-3f6d56625d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:28.958889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd39ec-f978-4d73-a4e9-3f6d56625d4a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:28.959301 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd39ec-f978-4d73-a4e9-3f6d56625d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:28.959704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd39ec-f978-4d73-a4e9-3f6d56625d4a None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:28.960352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd39ec-f978-4d73-a4e9-3f6d56625d4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:28.960655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd39ec-f978-4d73-a4e9-3f6d56625d4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:28.962568 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd39ec-f978-4d73-a4e9-3f6d56625d4a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:28.989180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 232/468] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:28.989180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd39ec-f978-4d73-a4e9-3f6d56625d4a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:29.013904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a87559-b283-4d55-9813-b8d65482c2c1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:29.016148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:29.051801 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2131e3-8973-47ed-bbea-ed6c43112d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:29.052468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2131e3-8973-47ed-bbea-ed6c43112d71 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:29.052692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2131e3-8973-47ed-bbea-ed6c43112d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:29.052966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2131e3-8973-47ed-bbea-ed6c43112d71 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:29.053690 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2131e3-8973-47ed-bbea-ed6c43112d71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:29.055168 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2131e3-8973-47ed-bbea-ed6c43112d71 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:29.089593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 237/469] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:15:28 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:15:29.090599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2131e3-8973-47ed-bbea-ed6c43112d71 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:29.098929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd39ec-f978-4d73-a4e9-3f6d56625d4a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:29.100813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:29.164927 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b85d7b-aa2c-4e41-9b26-baa9d6d21c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:29.165688 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b85d7b-aa2c-4e41-9b26-baa9d6d21c34 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:29.166105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b85d7b-aa2c-4e41-9b26-baa9d6d21c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:29.166450 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b85d7b-aa2c-4e41-9b26-baa9d6d21c34 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:29.167184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b85d7b-aa2c-4e41-9b26-baa9d6d21c34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:29.167558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b85d7b-aa2c-4e41-9b26-baa9d6d21c34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:29.169196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b85d7b-aa2c-4e41-9b26-baa9d6d21c34 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:29.182429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 233/470] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:29.188296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b85d7b-aa2c-4e41-9b26-baa9d6d21c34 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:29.222379 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2131e3-8973-47ed-bbea-ed6c43112d71 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:29.261805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef2ebc-4727-443b-8501-f4c6f7a4005d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:29.262582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef2ebc-4727-443b-8501-f4c6f7a4005d None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:29.262920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef2ebc-4727-443b-8501-f4c6f7a4005d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:29.263232 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef2ebc-4727-443b-8501-f4c6f7a4005d None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:29.264003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef2ebc-4727-443b-8501-f4c6f7a4005d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:29.265447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef2ebc-4727-443b-8501-f4c6f7a4005d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:29.292990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 238/471] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:15:29 2022] POST /identity/v3/users => generated 248 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:15:29.294390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef2ebc-4727-443b-8501-f4c6f7a4005d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:29.309536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b85d7b-aa2c-4e41-9b26-baa9d6d21c34 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:29.321944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:29.354797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d340d1-9ad4-41a0-966d-4a46cbb85d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:29.355445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d340d1-9ad4-41a0-966d-4a46cbb85d88 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:29.355724 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d340d1-9ad4-41a0-966d-4a46cbb85d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:29.355996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d340d1-9ad4-41a0-966d-4a46cbb85d88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:29.371234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d340d1-9ad4-41a0-966d-4a46cbb85d88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:29.374140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d340d1-9ad4-41a0-966d-4a46cbb85d88 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b3d340d1-9ad4-41a0-966d-4a46cbb85d88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:29.392479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:29.393532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:29.394363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:29.394363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:29.394363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:29.394523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 234/472] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:29.395757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e0364-0350-4138-89fc-2510ad73ea77 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:29.396040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e0364-0350-4138-89fc-2510ad73ea77 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:29.396312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e0364-0350-4138-89fc-2510ad73ea77 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:29.397807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 235/473] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:29.402941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e0364-0350-4138-89fc-2510ad73ea77 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:29.425029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:29.443562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef2ebc-4727-443b-8501-f4c6f7a4005d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:29.496341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b700906-113e-4a01-ab28-7e636c06bf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:29.497056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b700906-113e-4a01-ab28-7e636c06bf42 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:29.497369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b700906-113e-4a01-ab28-7e636c06bf42 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:29.497639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b700906-113e-4a01-ab28-7e636c06bf42 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:29.498351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b700906-113e-4a01-ab28-7e636c06bf42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:29.499809 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b700906-113e-4a01-ab28-7e636c06bf42 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:29.531407 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 239/474] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:15:29 2022] POST /identity/v3/users => generated 248 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:15:29.533627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e624b9-3da4-478d-b87e-cfd8aa46b0dc None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:29.533911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e624b9-3da4-478d-b87e-cfd8aa46b0dc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:29.534185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e624b9-3da4-478d-b87e-cfd8aa46b0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:29.539618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e0364-0350-4138-89fc-2510ad73ea77 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:29.556098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c8e624b9-3da4-478d-b87e-cfd8aa46b0dc None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:29.557999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c8e624b9-3da4-478d-b87e-cfd8aa46b0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:29.580111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704eb66-65fe-46e2-a0f2-25c6c672ad42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:29.581245 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704eb66-65fe-46e2-a0f2-25c6c672ad42 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:29.581758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704eb66-65fe-46e2-a0f2-25c6c672ad42 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:29.582218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704eb66-65fe-46e2-a0f2-25c6c672ad42 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:29.583133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704eb66-65fe-46e2-a0f2-25c6c672ad42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:29.583639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704eb66-65fe-46e2-a0f2-25c6c672ad42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:29.585287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704eb66-65fe-46e2-a0f2-25c6c672ad42 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:29.599612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 236/475] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:29.605806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704eb66-65fe-46e2-a0f2-25c6c672ad42 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:29.654260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 240/476] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:29.710099 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:29.741191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704eb66-65fe-46e2-a0f2-25c6c672ad42 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:29.781652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2a6b0-a555-4852-8e08-81949c420e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:29.782638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2a6b0-a555-4852-8e08-81949c420e43 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:29.783116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2a6b0-a555-4852-8e08-81949c420e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:29.783570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2a6b0-a555-4852-8e08-81949c420e43 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:29.784441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2a6b0-a555-4852-8e08-81949c420e43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:29.786093 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2a6b0-a555-4852-8e08-81949c420e43 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:29.811087 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 237/477] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:15:29 2022] POST /identity/v3/users => generated 249 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:15:30.097564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e624b9-3da4-478d-b87e-cfd8aa46b0dc None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:30.100373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:30.205384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e624b9-3da4-478d-b87e-cfd8aa46b0dc None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:30.254366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d70407-4812-40a5-8eae-ed92d8c28c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:30.254995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d70407-4812-40a5-8eae-ed92d8c28c25 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:30.255334 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d70407-4812-40a5-8eae-ed92d8c28c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:30.255608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d70407-4812-40a5-8eae-ed92d8c28c25 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:30.256291 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d70407-4812-40a5-8eae-ed92d8c28c25 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:30.257705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d70407-4812-40a5-8eae-ed92d8c28c25 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:30.272079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.sql.core [None req-42d70407-4812-40a5-8eae-ed92d8c28c25 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 17 04:15:30.272079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 17 04:15:30.272079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [parameters: {'id': '2de28eb41a0f49bfb16f33e20e79cfb6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 17 04:15:30.272079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81417) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-42d70407-4812-40a5-8eae-ed92d8c28c25 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application conn.query(q) Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result.read() Sep 17 04:15:30.292714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return ref.to_dict() Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 04:15:30.293682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application next(self.gen) Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application yield resource Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application next(self.gen) Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise self.value Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application session.commit() Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 04:15:30.294562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.session.flush() Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application compat.raise_( Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 04:15:30.295451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 04:15:30.296317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application conn.query(q) Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result.read() Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 04:15:30.298111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [parameters: {'id': '2de28eb41a0f49bfb16f33e20e79cfb6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:30.300260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 17 04:15:30.302250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:30.305744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 241/478] 10.222.0.60 () {60 vars in 1131 bytes} [Sat Sep 17 04:15:30 2022] POST /identity/v3/roles => generated 144 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:30.305881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2a6b0-a555-4852-8e08-81949c420e43 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:30.412714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2a6b0-a555-4852-8e08-81949c420e43 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:30.442678 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:30.456560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a1bfd-ffbe-4cf5-8d16-ff3d0190510d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:30.457653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a1bfd-ffbe-4cf5-8d16-ff3d0190510d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:30.458163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a1bfd-ffbe-4cf5-8d16-ff3d0190510d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:30.458639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a1bfd-ffbe-4cf5-8d16-ff3d0190510d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:30.469571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a1bfd-ffbe-4cf5-8d16-ff3d0190510d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-9d2a1bfd-ffbe-4cf5-8d16-ff3d0190510d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:30.475114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:30.475988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:30.476786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:30.476786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:30.476786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:30.476786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:30.476786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:30.476786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:30.476786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:30.476786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:30.476786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:30.476786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:30.476786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 04:15:30.476786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:30.479023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 238/479] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:15:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:30.484696 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d70407-4812-40a5-8eae-ed92d8c28c25 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:30.577754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:30.627031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d70407-4812-40a5-8eae-ed92d8c28c25 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:30.674735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6b28f-39d3-422b-ba1d-cb7e3d1bff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:30.675365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6b28f-39d3-422b-ba1d-cb7e3d1bff77 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:30.675631 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6b28f-39d3-422b-ba1d-cb7e3d1bff77 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:30.675897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6b28f-39d3-422b-ba1d-cb7e3d1bff77 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:30.676569 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6b28f-39d3-422b-ba1d-cb7e3d1bff77 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:30.676865 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6b28f-39d3-422b-ba1d-cb7e3d1bff77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:30.678276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6b28f-39d3-422b-ba1d-cb7e3d1bff77 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:30.687211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 242/480] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:31.527451 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddc541-ace8-4600-9671-69fccf38d096 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:31.528024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddc541-ace8-4600-9671-69fccf38d096 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:31.528024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddc541-ace8-4600-9671-69fccf38d096 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:31.529651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 239/481] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:31.535178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7231c3-785c-41f0-afde-cbaa9c1ef194 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:31.535578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7231c3-785c-41f0-afde-cbaa9c1ef194 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:31.535762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7231c3-785c-41f0-afde-cbaa9c1ef194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:31.562109 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dc7231c3-785c-41f0-afde-cbaa9c1ef194 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:31.563308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dc7231c3-785c-41f0-afde-cbaa9c1ef194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:31.651391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 243/482] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:32.044168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddc541-ace8-4600-9671-69fccf38d096 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:32.161315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddc541-ace8-4600-9671-69fccf38d096 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:32.194329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412cea7-c803-49ce-8eec-d5bf5e4f5282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:32.194954 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412cea7-c803-49ce-8eec-d5bf5e4f5282 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:32.195224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412cea7-c803-49ce-8eec-d5bf5e4f5282 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:32.195487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412cea7-c803-49ce-8eec-d5bf5e4f5282 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:32.206615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412cea7-c803-49ce-8eec-d5bf5e4f5282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-2412cea7-c803-49ce-8eec-d5bf5e4f5282 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:32.211360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:32.212234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:32.213118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 240/483] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:32.220026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7231c3-785c-41f0-afde-cbaa9c1ef194 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:32.247351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbdc24-e435-41b2-be8e-b346238eedbc None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:32.247619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbdc24-e435-41b2-be8e-b346238eedbc None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:32.247892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbdc24-e435-41b2-be8e-b346238eedbc None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:32.249529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 241/484] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:32.254632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c99b0-95e9-4c08-abfa-6f0fb8abd407 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:32.254920 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c99b0-95e9-4c08-abfa-6f0fb8abd407 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:32.255191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c99b0-95e9-4c08-abfa-6f0fb8abd407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:32.277076 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-aa1c99b0-95e9-4c08-abfa-6f0fb8abd407 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:32.278431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-aa1c99b0-95e9-4c08-abfa-6f0fb8abd407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:32.311981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:32.335521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:32.356282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 242/485] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:32.400167 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7231c3-785c-41f0-afde-cbaa9c1ef194 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:32.454953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0600e-f987-490d-b0b8-72d05076a09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:32.455603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0600e-f987-490d-b0b8-72d05076a09f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:32.455880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0600e-f987-490d-b0b8-72d05076a09f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:32.456144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0600e-f987-490d-b0b8-72d05076a09f None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:32.456813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0600e-f987-490d-b0b8-72d05076a09f None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:32.457133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0600e-f987-490d-b0b8-72d05076a09f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:32.458507 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0600e-f987-490d-b0b8-72d05076a09f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:32.467581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 244/486] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:32.472975 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c99b0-95e9-4c08-abfa-6f0fb8abd407 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:32.597480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c99b0-95e9-4c08-abfa-6f0fb8abd407 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:32.638242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0600e-f987-490d-b0b8-72d05076a09f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:32.642411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6d08c-7a3b-4b07-a3b7-624f12d2b217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:32.643075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6d08c-7a3b-4b07-a3b7-624f12d2b217 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:32.643345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6d08c-7a3b-4b07-a3b7-624f12d2b217 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:32.643617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6d08c-7a3b-4b07-a3b7-624f12d2b217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:32.673784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6d08c-7a3b-4b07-a3b7-624f12d2b217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:32.675302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6d08c-7a3b-4b07-a3b7-624f12d2b217 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-35e6d08c-7a3b-4b07-a3b7-624f12d2b217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:32.701489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:32.702470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:32.703293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:32.703293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:32.703293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:32.704027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 243/487] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:32.715599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6d08c-7a3b-4b07-a3b7-624f12d2b217 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:32.769483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:32.814105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0600e-f987-490d-b0b8-72d05076a09f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:32.833854 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:32.836305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6d08c-7a3b-4b07-a3b7-624f12d2b217 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:32.848400 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48205f71-95b0-4655-9638-8067e0e46eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:32.849045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48205f71-95b0-4655-9638-8067e0e46eac None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:32.849345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48205f71-95b0-4655-9638-8067e0e46eac None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:32.849610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48205f71-95b0-4655-9638-8067e0e46eac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:32.859721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48205f71-95b0-4655-9638-8067e0e46eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-48205f71-95b0-4655-9638-8067e0e46eac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:32.865196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:32.866064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:32.867217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 245/488] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:32.867861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5373e-1983-4755-a871-9965deaded3b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:32.868527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5373e-1983-4755-a871-9965deaded3b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:32.868527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5373e-1983-4755-a871-9965deaded3b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:32.873825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 246/489] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:32.873825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f9b54-6c04-4506-86c8-0dd1d754a5ed None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:32.873825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f9b54-6c04-4506-86c8-0dd1d754a5ed None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:32.873825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f9b54-6c04-4506-86c8-0dd1d754a5ed None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:32.873825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 247/490] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:32.873825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f9b54-6c04-4506-86c8-0dd1d754a5ed None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:32.881058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278079a4-0950-4139-98a8-93dcb7d80c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:32.881979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278079a4-0950-4139-98a8-93dcb7d80c50 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:32.882467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278079a4-0950-4139-98a8-93dcb7d80c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:32.882921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278079a4-0950-4139-98a8-93dcb7d80c50 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:32.883810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278079a4-0950-4139-98a8-93dcb7d80c50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:32.884211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278079a4-0950-4139-98a8-93dcb7d80c50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:32.885836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278079a4-0950-4139-98a8-93dcb7d80c50 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:32.897967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 244/491] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:32.899115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec47f28-6e05-408e-b62f-d3414b5fcf2a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:32.899388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec47f28-6e05-408e-b62f-d3414b5fcf2a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:32.899651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec47f28-6e05-408e-b62f-d3414b5fcf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:32.917683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-aec47f28-6e05-408e-b62f-d3414b5fcf2a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:32.918848 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-aec47f28-6e05-408e-b62f-d3414b5fcf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:32.985616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 245/492] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:32.987177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72857602-7b60-4066-b537-4a8d3cedad4b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:32.987460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72857602-7b60-4066-b537-4a8d3cedad4b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:32.987728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72857602-7b60-4066-b537-4a8d3cedad4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:32.994726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f9b54-6c04-4506-86c8-0dd1d754a5ed None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:33.008240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-72857602-7b60-4066-b537-4a8d3cedad4b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:33.009869 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-72857602-7b60-4066-b537-4a8d3cedad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:33.028461 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:33.033012 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680db31-42f6-4879-9919-340a2f90e417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:33.034118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680db31-42f6-4879-9919-340a2f90e417 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:33.034570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680db31-42f6-4879-9919-340a2f90e417 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:33.035120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680db31-42f6-4879-9919-340a2f90e417 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:33.036460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680db31-42f6-4879-9919-340a2f90e417 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:33.037482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680db31-42f6-4879-9919-340a2f90e417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:33.039652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680db31-42f6-4879-9919-340a2f90e417 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:33.050649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 248/493] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:33.051307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680db31-42f6-4879-9919-340a2f90e417 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:33.084213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 246/494] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:33.085114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72857602-7b60-4066-b537-4a8d3cedad4b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:33.106452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:33.166364 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680db31-42f6-4879-9919-340a2f90e417 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:33.206473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72857602-7b60-4066-b537-4a8d3cedad4b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:33.209404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d39fd5-4ded-4efa-af8e-4a02fb8523ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:33.210272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d39fd5-4ded-4efa-af8e-4a02fb8523ee None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:33.210730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d39fd5-4ded-4efa-af8e-4a02fb8523ee None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:33.211301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d39fd5-4ded-4efa-af8e-4a02fb8523ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:33.227593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d39fd5-4ded-4efa-af8e-4a02fb8523ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:33.229410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d39fd5-4ded-4efa-af8e-4a02fb8523ee None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-70d39fd5-4ded-4efa-af8e-4a02fb8523ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:33.247462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:33.248395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:33.249346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:33.249346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:33.249346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:33.249346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:33.249346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:33.249346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:33.249346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:33.249346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:33.249346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:15:33.249346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:33.249346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 249/495] 10.222.0.60 () {56 vars in 1093 bytes} [Sat Sep 17 04:15:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:33.254501 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d39fd5-4ded-4efa-af8e-4a02fb8523ee None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:33.261816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4f65c-b5e6-4011-aac7-ebdc8a5bfbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:33.262471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4f65c-b5e6-4011-aac7-ebdc8a5bfbfe None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:33.262744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4f65c-b5e6-4011-aac7-ebdc8a5bfbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:33.263008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4f65c-b5e6-4011-aac7-ebdc8a5bfbfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:33.276886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4f65c-b5e6-4011-aac7-ebdc8a5bfbfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:33.278391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4f65c-b5e6-4011-aac7-ebdc8a5bfbfe None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c4e4f65c-b5e6-4011-aac7-ebdc8a5bfbfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:33.312789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:33.313750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:33.314589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:33.314589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:33.314589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:33.314589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 247/496] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:33.320162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4f65c-b5e6-4011-aac7-ebdc8a5bfbfe None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:33.327530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:33.378909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d39fd5-4ded-4efa-af8e-4a02fb8523ee None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:33.381205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:33.426888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6882ea7-a23d-48c3-83b2-45d71b243693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:33.427501 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6882ea7-a23d-48c3-83b2-45d71b243693 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:33.427780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6882ea7-a23d-48c3-83b2-45d71b243693 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:33.428013 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4f65c-b5e6-4011-aac7-ebdc8a5bfbfe None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:33.428111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6882ea7-a23d-48c3-83b2-45d71b243693 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:33.428776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6882ea7-a23d-48c3-83b2-45d71b243693 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:33.429098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6882ea7-a23d-48c3-83b2-45d71b243693 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:33.431788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6882ea7-a23d-48c3-83b2-45d71b243693 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:33.466563 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fb0d5-085a-44f6-b6bc-ec76f85f733f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:33.467211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fb0d5-085a-44f6-b6bc-ec76f85f733f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:33.467490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fb0d5-085a-44f6-b6bc-ec76f85f733f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:33.467764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fb0d5-085a-44f6-b6bc-ec76f85f733f None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:33.468414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fb0d5-085a-44f6-b6bc-ec76f85f733f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:33.468719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fb0d5-085a-44f6-b6bc-ec76f85f733f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:33.470257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fb0d5-085a-44f6-b6bc-ec76f85f733f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:33.473373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 250/497] 10.222.0.60 () {56 vars in 1133 bytes} [Sat Sep 17 04:15:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:33.474481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6882ea7-a23d-48c3-83b2-45d71b243693 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:33.486819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 248/498] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:33.487916 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fb0d5-085a-44f6-b6bc-ec76f85f733f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:33.555582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:33.593689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fb0d5-085a-44f6-b6bc-ec76f85f733f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:33.600462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6882ea7-a23d-48c3-83b2-45d71b243693 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:33.630034 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08e7ed-c51f-4ea2-a09d-860fef7aaa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:33.630576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08e7ed-c51f-4ea2-a09d-860fef7aaa9a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:33.630820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08e7ed-c51f-4ea2-a09d-860fef7aaa9a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:33.631080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08e7ed-c51f-4ea2-a09d-860fef7aaa9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:33.638852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9c18a-34f1-4003-9b0e-012456bfdce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:33.640030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9c18a-34f1-4003-9b0e-012456bfdce2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:33.640499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9c18a-34f1-4003-9b0e-012456bfdce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:33.640958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9c18a-34f1-4003-9b0e-012456bfdce2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:33.641150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08e7ed-c51f-4ea2-a09d-860fef7aaa9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-4a08e7ed-c51f-4ea2-a09d-860fef7aaa9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:33.645775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:33.646668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:33.647476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 249/499] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:33.648130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08e7ed-c51f-4ea2-a09d-860fef7aaa9a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:33.654955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9c18a-34f1-4003-9b0e-012456bfdce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-60b9c18a-34f1-4003-9b0e-012456bfdce2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:33.659806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:33.660762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:33.661611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:33.661611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:33.661611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:33.661611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:33.661611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:33.661611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:33.661611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:33.661611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:33.661611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:33.661611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:33.661611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:33.661611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:33.665235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 251/500] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:33.666400 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9c18a-34f1-4003-9b0e-012456bfdce2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:33.692414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:33.750757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08e7ed-c51f-4ea2-a09d-860fef7aaa9a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:33.772484 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9c18a-34f1-4003-9b0e-012456bfdce2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:33.774208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:33.786979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff732b76-281e-4a27-937c-08c71ebcdf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:33.787731 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff732b76-281e-4a27-937c-08c71ebcdf8e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:33.788005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff732b76-281e-4a27-937c-08c71ebcdf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:33.788272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff732b76-281e-4a27-937c-08c71ebcdf8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:33.805392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5711e27-dcb5-44fb-b456-7d1d7e4919ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:33.806011 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5711e27-dcb5-44fb-b456-7d1d7e4919ae None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:33.806362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5711e27-dcb5-44fb-b456-7d1d7e4919ae None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:33.806625 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5711e27-dcb5-44fb-b456-7d1d7e4919ae None admin] PATH_INFO: `/v3/users/glance` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:33.813476 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5711e27-dcb5-44fb-b456-7d1d7e4919ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:33.814924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5711e27-dcb5-44fb-b456-7d1d7e4919ae None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e5711e27-dcb5-44fb-b456-7d1d7e4919ae None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:33.826326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:33.827332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:33.828225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:33.828225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:33.828225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:33.828225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:33.828225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:33.828225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:33.828225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:33.828225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:33.828225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 17 04:15:33.828225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:33.828225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 252/501] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:33.828780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b8060-7560-44ac-8966-57f113f6f00a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:33.828780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b8060-7560-44ac-8966-57f113f6f00a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:33.829010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b8060-7560-44ac-8966-57f113f6f00a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:33.830430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 253/502] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:33.830924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b8060-7560-44ac-8966-57f113f6f00a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:33.860848 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff732b76-281e-4a27-937c-08c71ebcdf8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:33.862369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff732b76-281e-4a27-937c-08c71ebcdf8e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:33.916825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:33.936949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b8060-7560-44ac-8966-57f113f6f00a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:33.967822 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 250/503] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:33.968736 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff732b76-281e-4a27-937c-08c71ebcdf8e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:33.973478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e3a2b-1199-4450-b180-f3bab75e6e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:33.974092 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e3a2b-1199-4450-b180-f3bab75e6e32 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:33.974367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e3a2b-1199-4450-b180-f3bab75e6e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:33.974635 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e3a2b-1199-4450-b180-f3bab75e6e32 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:33.975301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e3a2b-1199-4450-b180-f3bab75e6e32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:33.975597 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e3a2b-1199-4450-b180-f3bab75e6e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:33.976986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e3a2b-1199-4450-b180-f3bab75e6e32 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:33.985324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 254/504] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:33.986093 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e3a2b-1199-4450-b180-f3bab75e6e32 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:33.993926 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:34.067000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff732b76-281e-4a27-937c-08c71ebcdf8e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:34.082229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e3a2b-1199-4450-b180-f3bab75e6e32 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:34.099252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf169ea8-ecb5-4064-9354-fce8c6713739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:34.099857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf169ea8-ecb5-4064-9354-fce8c6713739 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:34.100127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf169ea8-ecb5-4064-9354-fce8c6713739 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:34.100388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf169ea8-ecb5-4064-9354-fce8c6713739 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:34.101042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf169ea8-ecb5-4064-9354-fce8c6713739 None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:34.101366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf169ea8-ecb5-4064-9354-fce8c6713739 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:34.102735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf169ea8-ecb5-4064-9354-fce8c6713739 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:34.115275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 251/505] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:34.115275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cf639-0f3a-4e0b-9881-06594577d4fb None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:34.115275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cf639-0f3a-4e0b-9881-06594577d4fb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:34.115275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cf639-0f3a-4e0b-9881-06594577d4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:34.118979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f553b-0bbb-4185-a0cc-e2ab9d1b5969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:34.119628 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f553b-0bbb-4185-a0cc-e2ab9d1b5969 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:34.119893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f553b-0bbb-4185-a0cc-e2ab9d1b5969 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:34.120160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f553b-0bbb-4185-a0cc-e2ab9d1b5969 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:34.120886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f553b-0bbb-4185-a0cc-e2ab9d1b5969 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:34.121222 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f553b-0bbb-4185-a0cc-e2ab9d1b5969 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:34.122649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f553b-0bbb-4185-a0cc-e2ab9d1b5969 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:34.130944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-551cf639-0f3a-4e0b-9881-06594577d4fb None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:34.132137 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-551cf639-0f3a-4e0b-9881-06594577d4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:34.136712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:34.140796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 255/506] 10.222.0.60 () {56 vars in 1137 bytes} [Sat Sep 17 04:15:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:34.142283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f553b-0bbb-4185-a0cc-e2ab9d1b5969 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:34.195568 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 252/507] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:34.196329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cf639-0f3a-4e0b-9881-06594577d4fb None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:34.215136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:34.240453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f553b-0bbb-4185-a0cc-e2ab9d1b5969 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:34.276808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15788e-fca7-4ecd-a427-9453c3a60c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:34.277471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15788e-fca7-4ecd-a427-9453c3a60c76 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:34.277742 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15788e-fca7-4ecd-a427-9453c3a60c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:34.277995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15788e-fca7-4ecd-a427-9453c3a60c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:34.289129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15788e-fca7-4ecd-a427-9453c3a60c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:34.290616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15788e-fca7-4ecd-a427-9453c3a60c76 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-1b15788e-fca7-4ecd-a427-9453c3a60c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:34.305457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:34.306435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:34.307388 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:34.307388 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:34.307388 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:34.309099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 256/508] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:34.310245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15788e-fca7-4ecd-a427-9453c3a60c76 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:34.315302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cf639-0f3a-4e0b-9881-06594577d4fb None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:34.353961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15b79e-3c23-43e3-a4e0-7cd9d0f6e659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:34.354587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15b79e-3c23-43e3-a4e0-7cd9d0f6e659 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:34.354862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15b79e-3c23-43e3-a4e0-7cd9d0f6e659 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:34.355129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15b79e-3c23-43e3-a4e0-7cd9d0f6e659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:34.365961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15b79e-3c23-43e3-a4e0-7cd9d0f6e659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:34.368268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15b79e-3c23-43e3-a4e0-7cd9d0f6e659 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:34.370318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-5e15b79e-3c23-43e3-a4e0-7cd9d0f6e659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:34.383289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:34.384200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:34.385123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:34.385123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:34.385123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:34.385123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 253/509] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:34.385358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15b79e-3c23-43e3-a4e0-7cd9d0f6e659 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:34.423227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15788e-fca7-4ecd-a427-9453c3a60c76 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:34.456352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:34.456498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dab33d-8a93-409b-af65-0cac0a7f769d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:34.457551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dab33d-8a93-409b-af65-0cac0a7f769d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:34.457551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dab33d-8a93-409b-af65-0cac0a7f769d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:34.457715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dab33d-8a93-409b-af65-0cac0a7f769d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:34.468905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dab33d-8a93-409b-af65-0cac0a7f769d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:34.470515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dab33d-8a93-409b-af65-0cac0a7f769d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f4dab33d-8a93-409b-af65-0cac0a7f769d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:34.484306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:34.485219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:34.486068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:34.486068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:34.486068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:34.486068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 257/510] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:34.486281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dab33d-8a93-409b-af65-0cac0a7f769d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:34.494106 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15b79e-3c23-43e3-a4e0-7cd9d0f6e659 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:34.527340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b18c67-a4dc-46a6-97db-dcac3b727bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:34.527937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b18c67-a4dc-46a6-97db-dcac3b727bdc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:34.528207 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b18c67-a4dc-46a6-97db-dcac3b727bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:34.528470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b18c67-a4dc-46a6-97db-dcac3b727bdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:34.591677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dab33d-8a93-409b-af65-0cac0a7f769d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:34.593957 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:34.594474 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b18c67-a4dc-46a6-97db-dcac3b727bdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:34.595930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b18c67-a4dc-46a6-97db-dcac3b727bdc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:34.626307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c8009-b0fa-43a6-98b2-f44b9a212a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:34.626914 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c8009-b0fa-43a6-98b2-f44b9a212a4e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:34.627182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c8009-b0fa-43a6-98b2-f44b9a212a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:34.627447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c8009-b0fa-43a6-98b2-f44b9a212a4e None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:34.628078 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c8009-b0fa-43a6-98b2-f44b9a212a4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:34.628368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c8009-b0fa-43a6-98b2-f44b9a212a4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:34.629768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c8009-b0fa-43a6-98b2-f44b9a212a4e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:34.641088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 258/511] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:34.641961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c8009-b0fa-43a6-98b2-f44b9a212a4e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:34.661606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:34.697849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 254/512] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:34.698682 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b18c67-a4dc-46a6-97db-dcac3b727bdc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:34.739362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c8009-b0fa-43a6-98b2-f44b9a212a4e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:34.772552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bca6dc-c710-46ea-ae83-67e813f0559e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:34.773202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bca6dc-c710-46ea-ae83-67e813f0559e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:34.773480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bca6dc-c710-46ea-ae83-67e813f0559e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:34.773749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bca6dc-c710-46ea-ae83-67e813f0559e None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:34.774750 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bca6dc-c710-46ea-ae83-67e813f0559e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:34.775158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bca6dc-c710-46ea-ae83-67e813f0559e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:34.776783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bca6dc-c710-46ea-ae83-67e813f0559e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:34.787534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 259/513] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:34.788303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bca6dc-c710-46ea-ae83-67e813f0559e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:34.800504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b18c67-a4dc-46a6-97db-dcac3b727bdc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:34.811590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:34.833605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865454b-6c92-420e-aea2-d92f45cee98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:34.834224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865454b-6c92-420e-aea2-d92f45cee98f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:34.834493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865454b-6c92-420e-aea2-d92f45cee98f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:34.834761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865454b-6c92-420e-aea2-d92f45cee98f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:34.844386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865454b-6c92-420e-aea2-d92f45cee98f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:34.845856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865454b-6c92-420e-aea2-d92f45cee98f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-1865454b-6c92-420e-aea2-d92f45cee98f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:34.859709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:34.865176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:34.866265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:34.866265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:34.866265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:34.866265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 255/514] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:34.866265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865454b-6c92-420e-aea2-d92f45cee98f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:34.882496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:34.889191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bca6dc-c710-46ea-ae83-67e813f0559e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:34.921667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df16a5-c915-4718-8754-2face7f615da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:34.922276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df16a5-c915-4718-8754-2face7f615da None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:34.922556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df16a5-c915-4718-8754-2face7f615da None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:34.922824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df16a5-c915-4718-8754-2face7f615da None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:34.923565 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df16a5-c915-4718-8754-2face7f615da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:34.923912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df16a5-c915-4718-8754-2face7f615da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:34.925343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df16a5-c915-4718-8754-2face7f615da None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:34.937489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 260/515] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:34.938277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df16a5-c915-4718-8754-2face7f615da None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:34.958108 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865454b-6c92-420e-aea2-d92f45cee98f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:34.990643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d62902-7dc9-42c2-bdf2-fd7acd79e978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:34.991211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d62902-7dc9-42c2-bdf2-fd7acd79e978 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:34.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d62902-7dc9-42c2-bdf2-fd7acd79e978 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:34.992130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d62902-7dc9-42c2-bdf2-fd7acd79e978 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:34.998577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d62902-7dc9-42c2-bdf2-fd7acd79e978 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:35.000010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d62902-7dc9-42c2-bdf2-fd7acd79e978 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-99d62902-7dc9-42c2-bdf2-fd7acd79e978 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:35.011643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:35.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:35.013333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:35.013333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:35.013333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:35.013333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:35.013333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:35.013333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:35.013333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:35.013333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:35.013333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 256/516] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:35.013812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d62902-7dc9-42c2-bdf2-fd7acd79e978 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:35.022033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:35.038873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df16a5-c915-4718-8754-2face7f615da None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:35.072874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752b067-daab-4c95-ba25-305621841cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:35.073530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752b067-daab-4c95-ba25-305621841cbf None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:35.073807 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752b067-daab-4c95-ba25-305621841cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:35.074076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752b067-daab-4c95-ba25-305621841cbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:35.083809 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752b067-daab-4c95-ba25-305621841cbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:35.085293 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752b067-daab-4c95-ba25-305621841cbf None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:35.090648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a752b067-daab-4c95-ba25-305621841cbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:35.099608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:35.100523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:35.106347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:35.106347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:35.106347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:35.106347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 261/517] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:35.106347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752b067-daab-4c95-ba25-305621841cbf None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:35.110261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d62902-7dc9-42c2-bdf2-fd7acd79e978 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:35.143741 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b929c-84a0-44b9-8914-52b9bd53af8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:35.144362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b929c-84a0-44b9-8914-52b9bd53af8d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:35.144636 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b929c-84a0-44b9-8914-52b9bd53af8d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:35.144902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b929c-84a0-44b9-8914-52b9bd53af8d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:35.152156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b929c-84a0-44b9-8914-52b9bd53af8d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:35.153630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b929c-84a0-44b9-8914-52b9bd53af8d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-921b929c-84a0-44b9-8914-52b9bd53af8d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:35.165725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:35.166831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:35.167767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:35.167767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:35.167767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:35.167767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:35.167767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:35.167767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:35.167767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:35.167767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:35.167767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 04:15:35.167767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:35.167767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 257/518] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:35.168337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b929c-84a0-44b9-8914-52b9bd53af8d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:35.201118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752b067-daab-4c95-ba25-305621841cbf None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:35.232824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fbd2a-fe3a-4fef-872a-64e14805dec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:35.233461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fbd2a-fe3a-4fef-872a-64e14805dec0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:35.233748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fbd2a-fe3a-4fef-872a-64e14805dec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:35.234020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fbd2a-fe3a-4fef-872a-64e14805dec0 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:35.234662 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fbd2a-fe3a-4fef-872a-64e14805dec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:35.234970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fbd2a-fe3a-4fef-872a-64e14805dec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:35.236386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fbd2a-fe3a-4fef-872a-64e14805dec0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:35.238630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:35.249172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 262/519] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:35.249921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fbd2a-fe3a-4fef-872a-64e14805dec0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:35.265511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b929c-84a0-44b9-8914-52b9bd53af8d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:35.298181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4f96f-81b9-458a-b923-8dc2be41d856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:35.298796 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4f96f-81b9-458a-b923-8dc2be41d856 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:35.299068 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4f96f-81b9-458a-b923-8dc2be41d856 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:35.299350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4f96f-81b9-458a-b923-8dc2be41d856 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:35.307017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4f96f-81b9-458a-b923-8dc2be41d856 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:35.308662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4f96f-81b9-458a-b923-8dc2be41d856 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:35.311170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-1cc4f96f-81b9-458a-b923-8dc2be41d856 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:35.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:35.321570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:35.322594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:35.322594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:35.322594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:35.322594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:35.322594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:35.322594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:35.322594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:35.322594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:35.322594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 04:15:35.322594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:35.326278 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 258/520] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:35.327047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4f96f-81b9-458a-b923-8dc2be41d856 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:35.345391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fbd2a-fe3a-4fef-872a-64e14805dec0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:35.379942 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136ebe4-ea37-42bd-ab84-a6fcaf1adf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:35.380549 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136ebe4-ea37-42bd-ab84-a6fcaf1adf4a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:35.380827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136ebe4-ea37-42bd-ab84-a6fcaf1adf4a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:35.381118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136ebe4-ea37-42bd-ab84-a6fcaf1adf4a None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:35.381818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136ebe4-ea37-42bd-ab84-a6fcaf1adf4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:35.382120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136ebe4-ea37-42bd-ab84-a6fcaf1adf4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:35.383552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136ebe4-ea37-42bd-ab84-a6fcaf1adf4a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:35.400652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 263/521] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:35.401805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136ebe4-ea37-42bd-ab84-a6fcaf1adf4a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:35.427950 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4f96f-81b9-458a-b923-8dc2be41d856 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:35.454281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:35.460009 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b87aef-655a-40bf-90a5-0049a0cf9959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:35.460617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b87aef-655a-40bf-90a5-0049a0cf9959 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:35.460893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b87aef-655a-40bf-90a5-0049a0cf9959 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:35.461196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b87aef-655a-40bf-90a5-0049a0cf9959 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:35.461841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b87aef-655a-40bf-90a5-0049a0cf9959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:35.462143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b87aef-655a-40bf-90a5-0049a0cf9959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:35.463574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b87aef-655a-40bf-90a5-0049a0cf9959 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:35.474855 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 259/522] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:35.475619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b87aef-655a-40bf-90a5-0049a0cf9959 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:35.498015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136ebe4-ea37-42bd-ab84-a6fcaf1adf4a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:35.532236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:35.532409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bae4c-bb06-42c8-a649-6d90029de71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:35.533017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bae4c-bb06-42c8-a649-6d90029de71e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:35.533323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bae4c-bb06-42c8-a649-6d90029de71e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:35.533593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bae4c-bb06-42c8-a649-6d90029de71e None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:35.534324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bae4c-bb06-42c8-a649-6d90029de71e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:35.534623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bae4c-bb06-42c8-a649-6d90029de71e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:35.536069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bae4c-bb06-42c8-a649-6d90029de71e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:35.552376 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 264/523] 10.222.0.60 () {56 vars in 1137 bytes} [Sat Sep 17 04:15:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:35.553292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bae4c-bb06-42c8-a649-6d90029de71e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:35.576133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b87aef-655a-40bf-90a5-0049a0cf9959 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:35.608974 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb4fdf-a125-44fc-b1fa-b51b6dca4596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:35.609846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb4fdf-a125-44fc-b1fa-b51b6dca4596 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:35.610280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb4fdf-a125-44fc-b1fa-b51b6dca4596 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:35.610755 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb4fdf-a125-44fc-b1fa-b51b6dca4596 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:35.611813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb4fdf-a125-44fc-b1fa-b51b6dca4596 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:35.613267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb4fdf-a125-44fc-b1fa-b51b6dca4596 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:35.635907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 260/524] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:15:35 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:15:35.636733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb4fdf-a125-44fc-b1fa-b51b6dca4596 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:35.656235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bae4c-bb06-42c8-a649-6d90029de71e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:35.680488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:35.706365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72caeabd-8670-4fde-9bbe-c360798014fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:35.706995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72caeabd-8670-4fde-9bbe-c360798014fa None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:35.707264 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72caeabd-8670-4fde-9bbe-c360798014fa None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:35.707531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72caeabd-8670-4fde-9bbe-c360798014fa None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:35.708296 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72caeabd-8670-4fde-9bbe-c360798014fa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:35.708591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72caeabd-8670-4fde-9bbe-c360798014fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:35.710080 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72caeabd-8670-4fde-9bbe-c360798014fa None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:35.729170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 265/525] 10.222.0.60 () {56 vars in 1139 bytes} [Sat Sep 17 04:15:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:35.730046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72caeabd-8670-4fde-9bbe-c360798014fa None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:35.762056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb4fdf-a125-44fc-b1fa-b51b6dca4596 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:35.780155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:35.800653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f27a0-d82d-43af-b2c2-32271bc46285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:35.801300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f27a0-d82d-43af-b2c2-32271bc46285 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:35.801577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f27a0-d82d-43af-b2c2-32271bc46285 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:35.801847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f27a0-d82d-43af-b2c2-32271bc46285 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:35.802596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f27a0-d82d-43af-b2c2-32271bc46285 None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=d3272d57d5d24e5a990a6d7c86e6d4bc) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:35.802943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f27a0-d82d-43af-b2c2-32271bc46285 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:35.804468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f27a0-d82d-43af-b2c2-32271bc46285 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:35.811433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 261/526] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:35 2022] GET /identity/v3/role_assignments?user.id=d3272d57d5d24e5a990a6d7c86e6d4bc&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:35.811433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f27a0-d82d-43af-b2c2-32271bc46285 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:35.854863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72caeabd-8670-4fde-9bbe-c360798014fa None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:35.905677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6c4c6-3f0f-4d69-8f40-fb6fa5e7426f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:35.906320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6c4c6-3f0f-4d69-8f40-fb6fa5e7426f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:35.906583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6c4c6-3f0f-4d69-8f40-fb6fa5e7426f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:35.906845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6c4c6-3f0f-4d69-8f40-fb6fa5e7426f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:35.915763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6c4c6-3f0f-4d69-8f40-fb6fa5e7426f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:35.917297 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6c4c6-3f0f-4d69-8f40-fb6fa5e7426f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d1f6c4c6-3f0f-4d69-8f40-fb6fa5e7426f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:35.931196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:35.932184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:35.933652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:35.933652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:35.933652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:35.933652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:35.933652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:35.933652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:35.933652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:35.933652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:35.933652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 266/527] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:35.933652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6c4c6-3f0f-4d69-8f40-fb6fa5e7426f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:35.968445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f27a0-d82d-43af-b2c2-32271bc46285 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:35.974552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:36.029622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd1219-549f-4264-8e7f-a8d50fca2da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:36.030647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd1219-549f-4264-8e7f-a8d50fca2da6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:36.031192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd1219-549f-4264-8e7f-a8d50fca2da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:36.031470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd1219-549f-4264-8e7f-a8d50fca2da6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:36.052432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6c4c6-3f0f-4d69-8f40-fb6fa5e7426f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:36.090280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:36.093450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914cb3c-c75d-4a0e-b190-dfb776d9c2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:36.094242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914cb3c-c75d-4a0e-b190-dfb776d9c2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:36.094680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914cb3c-c75d-4a0e-b190-dfb776d9c2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:36.095096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914cb3c-c75d-4a0e-b190-dfb776d9c2c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:36.111958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd1219-549f-4264-8e7f-a8d50fca2da6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:36.113307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd1219-549f-4264-8e7f-a8d50fca2da6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:36.171680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914cb3c-c75d-4a0e-b190-dfb776d9c2c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:36.173519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914cb3c-c75d-4a0e-b190-dfb776d9c2c1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:36.205981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:36.239897 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 262/528] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:36.240983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd1219-549f-4264-8e7f-a8d50fca2da6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:36.287291 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 267/529] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:36.292966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914cb3c-c75d-4a0e-b190-dfb776d9c2c1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:36.346832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:36.366289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd1219-549f-4264-8e7f-a8d50fca2da6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:36.402168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a5172-1194-4635-8dbc-82b9c0acd730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:36.403442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a5172-1194-4635-8dbc-82b9c0acd730 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:36.403864 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a5172-1194-4635-8dbc-82b9c0acd730 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:36.404563 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a5172-1194-4635-8dbc-82b9c0acd730 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:36.405445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a5172-1194-4635-8dbc-82b9c0acd730 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:36.406004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a5172-1194-4635-8dbc-82b9c0acd730 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:36.408170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a5172-1194-4635-8dbc-82b9c0acd730 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:36.409331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914cb3c-c75d-4a0e-b190-dfb776d9c2c1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:36.434173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 263/530] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:36.435404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a5172-1194-4635-8dbc-82b9c0acd730 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:36.441931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:36.444137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad735d0-df51-4a1f-8098-a2eb6a03a5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:36.444902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad735d0-df51-4a1f-8098-a2eb6a03a5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:36.445301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad735d0-df51-4a1f-8098-a2eb6a03a5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:36.445981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad735d0-df51-4a1f-8098-a2eb6a03a5b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:36.456734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad735d0-df51-4a1f-8098-a2eb6a03a5b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:36.458535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad735d0-df51-4a1f-8098-a2eb6a03a5b7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-bad735d0-df51-4a1f-8098-a2eb6a03a5b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:36.475179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:36.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:36.477362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:36.477362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:36.477362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:36.481320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 268/531] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:36.482325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad735d0-df51-4a1f-8098-a2eb6a03a5b7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:36.546473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a5172-1194-4635-8dbc-82b9c0acd730 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:36.582464 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c4215-eb83-4362-bbaf-f881d37a920f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:36.583235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad735d0-df51-4a1f-8098-a2eb6a03a5b7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:36.583595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c4215-eb83-4362-bbaf-f881d37a920f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:36.584081 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c4215-eb83-4362-bbaf-f881d37a920f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:36.584533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c4215-eb83-4362-bbaf-f881d37a920f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:36.593450 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:36.596443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c4215-eb83-4362-bbaf-f881d37a920f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:36.598686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c4215-eb83-4362-bbaf-f881d37a920f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-408c4215-eb83-4362-bbaf-f881d37a920f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:36.614903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:36.616025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:36.617396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:36.617396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:36.617396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:36.623164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bc6b5-7a08-41d5-9df9-e0b55fc45de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:36.623164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bc6b5-7a08-41d5-9df9-e0b55fc45de7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:36.623164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bc6b5-7a08-41d5-9df9-e0b55fc45de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:36.623164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bc6b5-7a08-41d5-9df9-e0b55fc45de7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:36.623519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bc6b5-7a08-41d5-9df9-e0b55fc45de7 None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=acf8a0ae03c94aa78cc2ab0e4e1c87b6) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:36.623519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bc6b5-7a08-41d5-9df9-e0b55fc45de7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:36.623519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bc6b5-7a08-41d5-9df9-e0b55fc45de7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:36.624175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 264/532] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:36.625286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c4215-eb83-4362-bbaf-f881d37a920f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:36.629157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 269/533] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:36 2022] GET /identity/v3/role_assignments?user.id=acf8a0ae03c94aa78cc2ab0e4e1c87b6&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:36.634766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bc6b5-7a08-41d5-9df9-e0b55fc45de7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:36.701481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:36.745916 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bc6b5-7a08-41d5-9df9-e0b55fc45de7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:36.796033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632500e-c595-4102-82c4-d421d05b7344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:36.796677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632500e-c595-4102-82c4-d421d05b7344 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:36.796957 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632500e-c595-4102-82c4-d421d05b7344 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:36.797254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632500e-c595-4102-82c4-d421d05b7344 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:36.797929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632500e-c595-4102-82c4-d421d05b7344 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:36.798231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632500e-c595-4102-82c4-d421d05b7344 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:36.799677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632500e-c595-4102-82c4-d421d05b7344 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:36.825058 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 270/534] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:36.832290 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c4215-eb83-4362-bbaf-f881d37a920f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:36.832811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632500e-c595-4102-82c4-d421d05b7344 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:36.873309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b49a-74c9-4a3b-bd47-dcad3d5c3f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:36.874322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b49a-74c9-4a3b-bd47-dcad3d5c3f51 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:36.874812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b49a-74c9-4a3b-bd47-dcad3d5c3f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:36.875282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b49a-74c9-4a3b-bd47-dcad3d5c3f51 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:36.876132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3b49a-74c9-4a3b-bd47-dcad3d5c3f51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:36.876625 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3b49a-74c9-4a3b-bd47-dcad3d5c3f51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:36.878289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3b49a-74c9-4a3b-bd47-dcad3d5c3f51 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:36.894162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 265/535] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:36.902080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b49a-74c9-4a3b-bd47-dcad3d5c3f51 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:36.976065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:37.016817 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632500e-c595-4102-82c4-d421d05b7344 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:37.037357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:37.068757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b49a-74c9-4a3b-bd47-dcad3d5c3f51 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:37.072765 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19d754-d5a4-4241-bb5c-824d6601e3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:37.073429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19d754-d5a4-4241-bb5c-824d6601e3ce None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:37.073698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19d754-d5a4-4241-bb5c-824d6601e3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:37.073963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19d754-d5a4-4241-bb5c-824d6601e3ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:37.087893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19d754-d5a4-4241-bb5c-824d6601e3ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:37.089396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19d754-d5a4-4241-bb5c-824d6601e3ce None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:37.108220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5f299-e980-46f9-aad1-2ef47175c8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:37.108859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5f299-e980-46f9-aad1-2ef47175c8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-fe19d754-d5a4-4241-bb5c-824d6601e3ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:37.108965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:37.109778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:37.111079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:37.111079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:37.111079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:37.111079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:37.111079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:37.111079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:37.111079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:37.111079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:37.111559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5f299-e980-46f9-aad1-2ef47175c8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:37.111559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5f299-e980-46f9-aad1-2ef47175c8d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:37.111701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 271/536] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:37.117961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19d754-d5a4-4241-bb5c-824d6601e3ce None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:37.121729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5f299-e980-46f9-aad1-2ef47175c8d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:37.123173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5f299-e980-46f9-aad1-2ef47175c8d7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d4a5f299-e980-46f9-aad1-2ef47175c8d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:37.134312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:37.135203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:37.136683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:37.136683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:37.136683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:37.136683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:37.136683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:37.136683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:37.136683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:37.136683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:37.136683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 266/537] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:37.141623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5f299-e980-46f9-aad1-2ef47175c8d7 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:37.242303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5f299-e980-46f9-aad1-2ef47175c8d7 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:37.257481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:37.278572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2367a3-8bf4-4d36-b240-a8a106fd6593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:37.279462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2367a3-8bf4-4d36-b240-a8a106fd6593 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:37.279824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2367a3-8bf4-4d36-b240-a8a106fd6593 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:37.280182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2367a3-8bf4-4d36-b240-a8a106fd6593 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:37.280971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2367a3-8bf4-4d36-b240-a8a106fd6593 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:37.281486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2367a3-8bf4-4d36-b240-a8a106fd6593 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:37.283038 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2367a3-8bf4-4d36-b240-a8a106fd6593 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:37.295579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 267/538] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:37.303845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2367a3-8bf4-4d36-b240-a8a106fd6593 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:37.321156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19d754-d5a4-4241-bb5c-824d6601e3ce None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:37.388412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e2fde-ef04-498a-8ba0-d5e437e93266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:37.389307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e2fde-ef04-498a-8ba0-d5e437e93266 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:37.389725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e2fde-ef04-498a-8ba0-d5e437e93266 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:37.390138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e2fde-ef04-498a-8ba0-d5e437e93266 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:37.391091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e2fde-ef04-498a-8ba0-d5e437e93266 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:37.391631 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e2fde-ef04-498a-8ba0-d5e437e93266 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:37.393250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e2fde-ef04-498a-8ba0-d5e437e93266 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:37.406391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2367a3-8bf4-4d36-b240-a8a106fd6593 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:37.418747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 272/539] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:37.425970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e2fde-ef04-498a-8ba0-d5e437e93266 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:37.446144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a0ce7-e7a6-41ee-a315-465b289e06c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:37.447092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a0ce7-e7a6-41ee-a315-465b289e06c5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:37.447469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a0ce7-e7a6-41ee-a315-465b289e06c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:37.447854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a0ce7-e7a6-41ee-a315-465b289e06c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:37.448780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a0ce7-e7a6-41ee-a315-465b289e06c5 None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=6adf758d5fb9430189ee9057b4b7ea97) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:37.449204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a0ce7-e7a6-41ee-a315-465b289e06c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:37.450709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a0ce7-e7a6-41ee-a315-465b289e06c5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:37.456817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 268/540] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:37 2022] GET /identity/v3/role_assignments?user.id=6adf758d5fb9430189ee9057b4b7ea97&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:37.477157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:37.576758 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e2fde-ef04-498a-8ba0-d5e437e93266 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:37.624369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79fdb9-c6d0-416d-b9af-32aad3941639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:37.625008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79fdb9-c6d0-416d-b9af-32aad3941639 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:37.625303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79fdb9-c6d0-416d-b9af-32aad3941639 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:37.625569 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79fdb9-c6d0-416d-b9af-32aad3941639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:37.626325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79fdb9-c6d0-416d-b9af-32aad3941639 None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=c9360be7ee9e4739b20c59955d28da47) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:37.626629 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79fdb9-c6d0-416d-b9af-32aad3941639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:37.628129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79fdb9-c6d0-416d-b9af-32aad3941639 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:37.634746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 273/541] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:37 2022] GET /identity/v3/role_assignments?user.id=c9360be7ee9e4739b20c59955d28da47&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:38.250530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ae205-df87-4a73-87c7-6826d7c60784 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:38.251075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ae205-df87-4a73-87c7-6826d7c60784 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:38.251075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ae205-df87-4a73-87c7-6826d7c60784 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:38.252424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 269/542] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:38.258072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251084d-6377-48ac-aff8-1552c203ee85 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:38.258492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251084d-6377-48ac-aff8-1552c203ee85 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:38.258681 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251084d-6377-48ac-aff8-1552c203ee85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:38.283186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c251084d-6377-48ac-aff8-1552c203ee85 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:38.284390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c251084d-6377-48ac-aff8-1552c203ee85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:38.290873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f02b77-bedf-4983-ba65-1c48c6ef550e None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:38.291149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f02b77-bedf-4983-ba65-1c48c6ef550e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:38.291429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f02b77-bedf-4983-ba65-1c48c6ef550e None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:38.293014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 270/543] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:38.299035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e901b5-e548-41a8-8ac3-93671a2f9c8e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:38.299565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e901b5-e548-41a8-8ac3-93671a2f9c8e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:38.299919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e901b5-e548-41a8-8ac3-93671a2f9c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:38.322637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e4e901b5-e548-41a8-8ac3-93671a2f9c8e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:38.324098 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e4e901b5-e548-41a8-8ac3-93671a2f9c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:38.360621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:38.399477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 274/544] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:38.404145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 271/545] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:38.784988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251084d-6377-48ac-aff8-1552c203ee85 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:38.803521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e901b5-e548-41a8-8ac3-93671a2f9c8e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:38.818666 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:38.920663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251084d-6377-48ac-aff8-1552c203ee85 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:38.960332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd6253-2555-4efd-b855-6b7a98a3fbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:38.961521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd6253-2555-4efd-b855-6b7a98a3fbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:38.962009 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd6253-2555-4efd-b855-6b7a98a3fbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:38.962470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd6253-2555-4efd-b855-6b7a98a3fbb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:38.975949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd6253-2555-4efd-b855-6b7a98a3fbb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-31dd6253-2555-4efd-b855-6b7a98a3fbb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:38.981511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:38.982414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:38.983224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:38.983224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:38.983224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:38.983224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:38.983224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:38.983224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:38.983224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:38.983224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:38.983224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:38.983224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:38.983224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:38.983224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:38.985963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 275/546] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:38.995290 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd6253-2555-4efd-b855-6b7a98a3fbb1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:39.001373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e901b5-e548-41a8-8ac3-93671a2f9c8e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:39.061454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d11af-62f1-4a8b-9a58-f76470bcd568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:39.062099 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d11af-62f1-4a8b-9a58-f76470bcd568 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:39.062376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d11af-62f1-4a8b-9a58-f76470bcd568 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:39.062647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d11af-62f1-4a8b-9a58-f76470bcd568 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:39.073457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d11af-62f1-4a8b-9a58-f76470bcd568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c04d11af-62f1-4a8b-9a58-f76470bcd568 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:39.079795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:39.080664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:39.081514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:39.081514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:39.081514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:39.081514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:39.081514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:39.081514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:39.081514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:39.081514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:39.081514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:39.081514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:39.081514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:39.081514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:39.082126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 272/547] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:39.086047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54152360-1ca5-4ff0-8f5d-5b9af2fdb261 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:39.086047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54152360-1ca5-4ff0-8f5d-5b9af2fdb261 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:39.086047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54152360-1ca5-4ff0-8f5d-5b9af2fdb261 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:39.089162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 273/548] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:39.089948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54152360-1ca5-4ff0-8f5d-5b9af2fdb261 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:39.114656 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:39.123448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:39.137512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd6253-2555-4efd-b855-6b7a98a3fbb1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:39.174569 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adbdd4-63fc-48d1-92e3-06d6842da3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:39.175600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adbdd4-63fc-48d1-92e3-06d6842da3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:39.176052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adbdd4-63fc-48d1-92e3-06d6842da3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:39.176494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adbdd4-63fc-48d1-92e3-06d6842da3f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:39.177551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adbdd4-63fc-48d1-92e3-06d6842da3f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:39.178094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adbdd4-63fc-48d1-92e3-06d6842da3f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:39.179709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adbdd4-63fc-48d1-92e3-06d6842da3f0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:39.188890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 276/549] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:39.190385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f24f7-2ec2-4fb1-93d8-523667824cd9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:39.190843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f24f7-2ec2-4fb1-93d8-523667824cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:39.191330 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f24f7-2ec2-4fb1-93d8-523667824cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:39.211038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a35f24f7-2ec2-4fb1-93d8-523667824cd9 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:39.212736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a35f24f7-2ec2-4fb1-93d8-523667824cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:39.217907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54152360-1ca5-4ff0-8f5d-5b9af2fdb261 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:39.264495 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fb2c5-cce1-4a8a-81e0-39d64ba9f1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:39.265410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fb2c5-cce1-4a8a-81e0-39d64ba9f1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:39.265728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fb2c5-cce1-4a8a-81e0-39d64ba9f1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:39.266300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fb2c5-cce1-4a8a-81e0-39d64ba9f1d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:39.267384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fb2c5-cce1-4a8a-81e0-39d64ba9f1d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:39.267692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fb2c5-cce1-4a8a-81e0-39d64ba9f1d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:39.269199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fb2c5-cce1-4a8a-81e0-39d64ba9f1d5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:39.285455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 274/550] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:39.286348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fb2c5-cce1-4a8a-81e0-39d64ba9f1d5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:39.286506 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 277/551] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:39.295197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f24f7-2ec2-4fb1-93d8-523667824cd9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:39.423818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:39.437656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:39.459341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fb2c5-cce1-4a8a-81e0-39d64ba9f1d5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:39.472642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f24f7-2ec2-4fb1-93d8-523667824cd9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:39.519250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce2f15-6520-4eb1-a36c-3dd95f6e5145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:39.520141 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce2f15-6520-4eb1-a36c-3dd95f6e5145 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:39.520705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce2f15-6520-4eb1-a36c-3dd95f6e5145 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:39.521135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce2f15-6520-4eb1-a36c-3dd95f6e5145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:39.534879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8929f0-4d70-4fb8-be1f-57af7e761f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:39.535770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8929f0-4d70-4fb8-be1f-57af7e761f63 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:39.536200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8929f0-4d70-4fb8-be1f-57af7e761f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:39.536636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8929f0-4d70-4fb8-be1f-57af7e761f63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:39.538086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce2f15-6520-4eb1-a36c-3dd95f6e5145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:39.540258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce2f15-6520-4eb1-a36c-3dd95f6e5145 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:39.547792 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8929f0-4d70-4fb8-be1f-57af7e761f63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:39.549419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8929f0-4d70-4fb8-be1f-57af7e761f63 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ca8929f0-4d70-4fb8-be1f-57af7e761f63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:39.564401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:39.565564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:39.566689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:39.566689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:39.566689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:39.568353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 278/552] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-63ce2f15-6520-4eb1-a36c-3dd95f6e5145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:39.573157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:39.574244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:39.575107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:39.575107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:39.575107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:39.575107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 275/553] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:39.576670 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8929f0-4d70-4fb8-be1f-57af7e761f63 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:39.577257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce2f15-6520-4eb1-a36c-3dd95f6e5145 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:39.711723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8929f0-4d70-4fb8-be1f-57af7e761f63 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:39.718324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce2f15-6520-4eb1-a36c-3dd95f6e5145 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:39.743125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:39.754028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fb758-7cb2-44a5-81cf-a68969debce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:39.754939 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fb758-7cb2-44a5-81cf-a68969debce1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:39.755394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fb758-7cb2-44a5-81cf-a68969debce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:39.755840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fb758-7cb2-44a5-81cf-a68969debce1 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:39.756141 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955018f5-6100-43c8-be40-3390eb1c0f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:39.756810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955018f5-6100-43c8-be40-3390eb1c0f4f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:39.757109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955018f5-6100-43c8-be40-3390eb1c0f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:39.757287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fb758-7cb2-44a5-81cf-a68969debce1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:39.757383 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955018f5-6100-43c8-be40-3390eb1c0f4f None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:39.757832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fb758-7cb2-44a5-81cf-a68969debce1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:39.758053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955018f5-6100-43c8-be40-3390eb1c0f4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:39.758360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955018f5-6100-43c8-be40-3390eb1c0f4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:39.759849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955018f5-6100-43c8-be40-3390eb1c0f4f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:39.759970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fb758-7cb2-44a5-81cf-a68969debce1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:39.769249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:39.777192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 276/554] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:39.777192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955018f5-6100-43c8-be40-3390eb1c0f4f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:39.780454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 279/555] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:39.781515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fb758-7cb2-44a5-81cf-a68969debce1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:39.884878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955018f5-6100-43c8-be40-3390eb1c0f4f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:39.886285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fb758-7cb2-44a5-81cf-a68969debce1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:39.921174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c80f3-b400-4ee8-aff8-7d8e3c223c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:39.921811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c80f3-b400-4ee8-aff8-7d8e3c223c94 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:39.922086 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c80f3-b400-4ee8-aff8-7d8e3c223c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:39.922354 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c80f3-b400-4ee8-aff8-7d8e3c223c94 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:39.929339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcf585-5fd0-40d2-a8d1-7a21d18011e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:39.930323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c80f3-b400-4ee8-aff8-7d8e3c223c94 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:39.930435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcf585-5fd0-40d2-a8d1-7a21d18011e6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:39.930874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcf585-5fd0-40d2-a8d1-7a21d18011e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:39.931386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcf585-5fd0-40d2-a8d1-7a21d18011e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:39.932109 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c80f3-b400-4ee8-aff8-7d8e3c223c94 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:39.943456 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcf585-5fd0-40d2-a8d1-7a21d18011e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b48c80f3-b400-4ee8-aff8-7d8e3c223c94 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:39.944172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:39.945064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:39.946015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:39.946015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:39.946015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:39.946015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:39.946015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:39.946015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:39.946015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:39.946015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:39.946015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 04:15:39.946015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:39.950271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 280/556] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:15:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:39.951176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c80f3-b400-4ee8-aff8-7d8e3c223c94 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-9edcf585-5fd0-40d2-a8d1-7a21d18011e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:39.952128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:39.953483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:39.954386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 277/557] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:39.961797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcf585-5fd0-40d2-a8d1-7a21d18011e6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:40.021705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:40.039763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:40.090504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c80f3-b400-4ee8-aff8-7d8e3c223c94 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:40.091883 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcf585-5fd0-40d2-a8d1-7a21d18011e6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:40.134753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cbcac-45b4-4a57-9e2e-4d5ea0f8ec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:40.135915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cbcac-45b4-4a57-9e2e-4d5ea0f8ec38 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.136969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cbcac-45b4-4a57-9e2e-4d5ea0f8ec38 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.137623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cbcac-45b4-4a57-9e2e-4d5ea0f8ec38 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.142212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ab7e7-d477-40e5-b1fc-f205008c5bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:40.142821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ab7e7-d477-40e5-b1fc-f205008c5bde None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.143095 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ab7e7-d477-40e5-b1fc-f205008c5bde None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.143363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ab7e7-d477-40e5-b1fc-f205008c5bde None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.144155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ab7e7-d477-40e5-b1fc-f205008c5bde None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:40.144463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ab7e7-d477-40e5-b1fc-f205008c5bde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:40.145948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ab7e7-d477-40e5-b1fc-f205008c5bde None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:40.147788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cbcac-45b4-4a57-9e2e-4d5ea0f8ec38 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:40.149647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cbcac-45b4-4a57-9e2e-4d5ea0f8ec38 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d53cbcac-45b4-4a57-9e2e-4d5ea0f8ec38 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:40.163089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:40.164208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:40.165199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:40.165199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:40.165199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:40.165199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:40.165199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:40.165199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:40.165199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:40.165199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:40.165199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:15:40.165199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:40.168106 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 281/558] 10.222.0.60 () {56 vars in 1093 bytes} [Sat Sep 17 04:15:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:40.169353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cbcac-45b4-4a57-9e2e-4d5ea0f8ec38 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:40.197545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 278/559] 10.222.0.60 () {56 vars in 1135 bytes} [Sat Sep 17 04:15:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:40.198214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ab7e7-d477-40e5-b1fc-f205008c5bde None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:40.281150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cbcac-45b4-4a57-9e2e-4d5ea0f8ec38 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:40.289571 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:40.301978 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ab7e7-d477-40e5-b1fc-f205008c5bde None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:40.317709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c11547-ab1d-43f2-b1c0-713bf9a7fe3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:40.318329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c11547-ab1d-43f2-b1c0-713bf9a7fe3f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.318603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c11547-ab1d-43f2-b1c0-713bf9a7fe3f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.318876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c11547-ab1d-43f2-b1c0-713bf9a7fe3f None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.319205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:40.319771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c11547-ab1d-43f2-b1c0-713bf9a7fe3f None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:40.319878 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c11547-ab1d-43f2-b1c0-713bf9a7fe3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:40.321347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c11547-ab1d-43f2-b1c0-713bf9a7fe3f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:40.329266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 282/560] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:40.330417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd6ee5-42bb-49ac-bb3c-40cd44023d2f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.330705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd6ee5-42bb-49ac-bb3c-40cd44023d2f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.330969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd6ee5-42bb-49ac-bb3c-40cd44023d2f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.332161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 283/561] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:40.332850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd6ee5-42bb-49ac-bb3c-40cd44023d2f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:40.340368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980b24e-c36d-4cb6-9932-ef4118eaad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:40.340978 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980b24e-c36d-4cb6-9932-ef4118eaad24 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.341284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980b24e-c36d-4cb6-9932-ef4118eaad24 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.341552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980b24e-c36d-4cb6-9932-ef4118eaad24 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.342284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980b24e-c36d-4cb6-9932-ef4118eaad24 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:40.342586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980b24e-c36d-4cb6-9932-ef4118eaad24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:40.344052 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980b24e-c36d-4cb6-9932-ef4118eaad24 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:40.360835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 279/562] 10.222.0.60 () {56 vars in 1133 bytes} [Sat Sep 17 04:15:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:40.362347 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9f961-c9fb-4f14-bde9-fd8f287ddc3e None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.362588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9f961-c9fb-4f14-bde9-fd8f287ddc3e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.362853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9f961-c9fb-4f14-bde9-fd8f287ddc3e None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.364149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 280/563] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:40.369225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9f961-c9fb-4f14-bde9-fd8f287ddc3e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:40.436611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd6ee5-42bb-49ac-bb3c-40cd44023d2f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:40.465942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9f961-c9fb-4f14-bde9-fd8f287ddc3e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:40.469662 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c98821-ba9c-4cf7-8d85-8d7271d6c88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:40.470273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c98821-ba9c-4cf7-8d85-8d7271d6c88a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.470546 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c98821-ba9c-4cf7-8d85-8d7271d6c88a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.470811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c98821-ba9c-4cf7-8d85-8d7271d6c88a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.499374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd0892-9497-4ee6-b49d-45c3ebdd3154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:40.499986 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd0892-9497-4ee6-b49d-45c3ebdd3154 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.500262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd0892-9497-4ee6-b49d-45c3ebdd3154 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.500524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd0892-9497-4ee6-b49d-45c3ebdd3154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.504347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:40.510267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd0892-9497-4ee6-b49d-45c3ebdd3154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:40.511705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd0892-9497-4ee6-b49d-45c3ebdd3154 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-bbcd0892-9497-4ee6-b49d-45c3ebdd3154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:40.526004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:40.527066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:40.527985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:40.527985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:40.527985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:40.527985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 281/564] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:40.528208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71898d43-f46b-430f-bb29-387a0817f593 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.528564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71898d43-f46b-430f-bb29-387a0817f593 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.528727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71898d43-f46b-430f-bb29-387a0817f593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.541708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c98821-ba9c-4cf7-8d85-8d7271d6c88a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:40.543188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c98821-ba9c-4cf7-8d85-8d7271d6c88a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:40.545833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-71898d43-f46b-430f-bb29-387a0817f593 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:40.547003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-71898d43-f46b-430f-bb29-387a0817f593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:40.569823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:40.613866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 282/565] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:40.614701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71898d43-f46b-430f-bb29-387a0817f593 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:40.649286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 284/566] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:40.650685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42adf60-706e-48cb-9a14-f6411145f725 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.650960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42adf60-706e-48cb-9a14-f6411145f725 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.651316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42adf60-706e-48cb-9a14-f6411145f725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.669183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d42adf60-706e-48cb-9a14-f6411145f725 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:40.670382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d42adf60-706e-48cb-9a14-f6411145f725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:40.724973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71898d43-f46b-430f-bb29-387a0817f593 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:40.732743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:40.739442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 285/567] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:40.740542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42adf60-706e-48cb-9a14-f6411145f725 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:40.770932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfb579-4533-4963-8aa0-3353eadfacfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:40.772406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfb579-4533-4963-8aa0-3353eadfacfd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.772955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfb579-4533-4963-8aa0-3353eadfacfd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.773393 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfb579-4533-4963-8aa0-3353eadfacfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.786427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfb579-4533-4963-8aa0-3353eadfacfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:40.788763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfb579-4533-4963-8aa0-3353eadfacfd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-62bfb579-4533-4963-8aa0-3353eadfacfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:40.810239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:40.811389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:40.812510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:40.812510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:40.812510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:40.815438 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 283/568] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:40.816375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfb579-4533-4963-8aa0-3353eadfacfd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:40.838535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:40.855251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42adf60-706e-48cb-9a14-f6411145f725 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:40.898057 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34abef-bbc7-41fe-8128-8d176af69311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:40.898659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34abef-bbc7-41fe-8128-8d176af69311 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.898961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34abef-bbc7-41fe-8128-8d176af69311 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.899196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34abef-bbc7-41fe-8128-8d176af69311 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.900371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34abef-bbc7-41fe-8128-8d176af69311 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:40.900371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34abef-bbc7-41fe-8128-8d176af69311 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:40.901673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34abef-bbc7-41fe-8128-8d176af69311 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:40.913237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 286/569] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:40.913631 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34abef-bbc7-41fe-8128-8d176af69311 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:40.931087 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfb579-4533-4963-8aa0-3353eadfacfd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:40.964226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70175b5b-18a8-414d-9e54-8c3e12417cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:40.964854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70175b5b-18a8-414d-9e54-8c3e12417cef None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:40.965173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70175b5b-18a8-414d-9e54-8c3e12417cef None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:40.965442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70175b5b-18a8-414d-9e54-8c3e12417cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:40.973455 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:40.975998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70175b5b-18a8-414d-9e54-8c3e12417cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:40.977496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70175b5b-18a8-414d-9e54-8c3e12417cef None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-70175b5b-18a8-414d-9e54-8c3e12417cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:40.991489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:40.992824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:40.993774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:40.993774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:40.993774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:40.993774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 284/570] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:40.993774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70175b5b-18a8-414d-9e54-8c3e12417cef None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.020429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34abef-bbc7-41fe-8128-8d176af69311 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.053033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:41.053934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a36e60-71e7-47a0-8d89-eb7b74c39176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.054545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a36e60-71e7-47a0-8d89-eb7b74c39176 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.054818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a36e60-71e7-47a0-8d89-eb7b74c39176 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.055086 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a36e60-71e7-47a0-8d89-eb7b74c39176 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.055720 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a36e60-71e7-47a0-8d89-eb7b74c39176 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:41.056039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a36e60-71e7-47a0-8d89-eb7b74c39176 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:41.057477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a36e60-71e7-47a0-8d89-eb7b74c39176 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:41.069618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 287/571] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:41.070334 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a36e60-71e7-47a0-8d89-eb7b74c39176 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.091833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70175b5b-18a8-414d-9e54-8c3e12417cef None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.126419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a09f7-918e-4f3a-b43d-b80be2b35cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.127053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a09f7-918e-4f3a-b43d-b80be2b35cee None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.127332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a09f7-918e-4f3a-b43d-b80be2b35cee None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.127603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a09f7-918e-4f3a-b43d-b80be2b35cee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.137737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a09f7-918e-4f3a-b43d-b80be2b35cee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-859a09f7-918e-4f3a-b43d-b80be2b35cee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:41.143102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:41.144204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 285/572] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:41.145399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a09f7-918e-4f3a-b43d-b80be2b35cee None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.172124 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a36e60-71e7-47a0-8d89-eb7b74c39176 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.188766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:41.205867 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9b799-1fd5-4d21-ba73-ec264720566b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.206495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9b799-1fd5-4d21-ba73-ec264720566b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.206770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9b799-1fd5-4d21-ba73-ec264720566b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.207041 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9b799-1fd5-4d21-ba73-ec264720566b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.213847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9b799-1fd5-4d21-ba73-ec264720566b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:41.215286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9b799-1fd5-4d21-ba73-ec264720566b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-47a9b799-1fd5-4d21-ba73-ec264720566b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:41.226440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:41.227498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:41.228514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:41.228514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:41.228514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:41.228514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:41.228514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:41.228514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:41.228514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:41.228514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:41.228514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 17 04:15:41.228514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:41.228514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 288/573] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:41.228514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9b799-1fd5-4d21-ba73-ec264720566b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.241359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a09f7-918e-4f3a-b43d-b80be2b35cee None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.263440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:41.275318 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b48693-ae1c-42b6-a1ef-f9bd80112a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.275918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b48693-ae1c-42b6-a1ef-f9bd80112a7f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.276207 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b48693-ae1c-42b6-a1ef-f9bd80112a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.276471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b48693-ae1c-42b6-a1ef-f9bd80112a7f None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.277109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b48693-ae1c-42b6-a1ef-f9bd80112a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:41.277428 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b48693-ae1c-42b6-a1ef-f9bd80112a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:41.278779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b48693-ae1c-42b6-a1ef-f9bd80112a7f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:41.290078 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 286/574] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:41.290970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b48693-ae1c-42b6-a1ef-f9bd80112a7f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.325394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9b799-1fd5-4d21-ba73-ec264720566b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.357439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48511b95-6f55-4bdd-85af-46062ccd7fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.358062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48511b95-6f55-4bdd-85af-46062ccd7fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.358345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48511b95-6f55-4bdd-85af-46062ccd7fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.358610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48511b95-6f55-4bdd-85af-46062ccd7fa9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.369018 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48511b95-6f55-4bdd-85af-46062ccd7fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-48511b95-6f55-4bdd-85af-46062ccd7fa9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:41.374343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:41.375273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:41.376113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 289/575] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:41.376798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48511b95-6f55-4bdd-85af-46062ccd7fa9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.395761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b48693-ae1c-42b6-a1ef-f9bd80112a7f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.417260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:41.429537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5c4eb-ae61-49a7-bc5a-fbc58d54415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.430171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5c4eb-ae61-49a7-bc5a-fbc58d54415a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.430472 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5c4eb-ae61-49a7-bc5a-fbc58d54415a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.430718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5c4eb-ae61-49a7-bc5a-fbc58d54415a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.437288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5c4eb-ae61-49a7-bc5a-fbc58d54415a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:41.438737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5c4eb-ae61-49a7-bc5a-fbc58d54415a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-61d5c4eb-ae61-49a7-bc5a-fbc58d54415a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:41.450545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:41.451614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:41.452521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:41.452521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:41.452521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:41.452521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:41.452521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:41.452521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:41.452521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:41.452521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:41.452521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 287/576] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:41.452521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5c4eb-ae61-49a7-bc5a-fbc58d54415a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.478860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48511b95-6f55-4bdd-85af-46062ccd7fa9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.494756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:41.513877 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01dafb-0e0b-45c7-be6b-35d7ff5b7b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.514481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01dafb-0e0b-45c7-be6b-35d7ff5b7b43 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.514788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01dafb-0e0b-45c7-be6b-35d7ff5b7b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.515054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01dafb-0e0b-45c7-be6b-35d7ff5b7b43 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.515713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01dafb-0e0b-45c7-be6b-35d7ff5b7b43 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:41.516014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01dafb-0e0b-45c7-be6b-35d7ff5b7b43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:41.517426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01dafb-0e0b-45c7-be6b-35d7ff5b7b43 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:41.525267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 290/577] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:41.526045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01dafb-0e0b-45c7-be6b-35d7ff5b7b43 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.553909 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5c4eb-ae61-49a7-bc5a-fbc58d54415a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.587362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e47ed7-2879-43ed-9213-0979a6ca5579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.587987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e47ed7-2879-43ed-9213-0979a6ca5579 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.588260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e47ed7-2879-43ed-9213-0979a6ca5579 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.588525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e47ed7-2879-43ed-9213-0979a6ca5579 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.589266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e47ed7-2879-43ed-9213-0979a6ca5579 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:41.589567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e47ed7-2879-43ed-9213-0979a6ca5579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:41.590982 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e47ed7-2879-43ed-9213-0979a6ca5579 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:41.608869 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 288/578] 10.222.0.60 () {56 vars in 1137 bytes} [Sat Sep 17 04:15:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:41.609869 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e47ed7-2879-43ed-9213-0979a6ca5579 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.626372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:41.629352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01dafb-0e0b-45c7-be6b-35d7ff5b7b43 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.668290 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a603ad-6601-4a34-a626-93d8e65a6093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.669147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a603ad-6601-4a34-a626-93d8e65a6093 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.669276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a603ad-6601-4a34-a626-93d8e65a6093 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.669521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a603ad-6601-4a34-a626-93d8e65a6093 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.676382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a603ad-6601-4a34-a626-93d8e65a6093 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:41.677871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a603ad-6601-4a34-a626-93d8e65a6093 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-27a603ad-6601-4a34-a626-93d8e65a6093 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:41.688997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:41.690013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:41.690901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:41.690901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:41.690901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:41.690901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:41.690901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:41.690901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:41.690901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:41.690901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:41.690901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 291/579] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:41.690901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a603ad-6601-4a34-a626-93d8e65a6093 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.709266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:41.712328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e47ed7-2879-43ed-9213-0979a6ca5579 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.745863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c05b86-f9be-44ff-9a86-b71821949886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.746496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c05b86-f9be-44ff-9a86-b71821949886 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.746770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c05b86-f9be-44ff-9a86-b71821949886 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.747035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c05b86-f9be-44ff-9a86-b71821949886 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.747693 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c05b86-f9be-44ff-9a86-b71821949886 None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:41.747987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c05b86-f9be-44ff-9a86-b71821949886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:41.749434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c05b86-f9be-44ff-9a86-b71821949886 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:41.757662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 289/580] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:41.758434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c05b86-f9be-44ff-9a86-b71821949886 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.795640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a603ad-6601-4a34-a626-93d8e65a6093 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.828038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f27ac-75a4-460e-9d51-30e8c5c5d660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.828642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f27ac-75a4-460e-9d51-30e8c5c5d660 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.828915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f27ac-75a4-460e-9d51-30e8c5c5d660 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.829211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f27ac-75a4-460e-9d51-30e8c5c5d660 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.829896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f27ac-75a4-460e-9d51-30e8c5c5d660 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:41.830194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f27ac-75a4-460e-9d51-30e8c5c5d660 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:41.831619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f27ac-75a4-460e-9d51-30e8c5c5d660 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:41.848429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 292/581] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:41.849280 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f27ac-75a4-460e-9d51-30e8c5c5d660 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.864527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c05b86-f9be-44ff-9a86-b71821949886 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.871323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:41.897813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd254a-2216-400e-a6c2-793833e5fdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.898435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd254a-2216-400e-a6c2-793833e5fdb5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.898705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd254a-2216-400e-a6c2-793833e5fdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.898971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd254a-2216-400e-a6c2-793833e5fdb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.908959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd254a-2216-400e-a6c2-793833e5fdb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:41.910416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd254a-2216-400e-a6c2-793833e5fdb5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-5ecd254a-2216-400e-a6c2-793833e5fdb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:41.923900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:41.924845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:41.925790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:41.925790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:41.925790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:41.925790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 290/582] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:41.925790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd254a-2216-400e-a6c2-793833e5fdb5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:41.940383 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:41.951098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f27ac-75a4-460e-9d51-30e8c5c5d660 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:41.983307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4044e3-d77a-4adc-be31-344d9ad123c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:41.983929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4044e3-d77a-4adc-be31-344d9ad123c3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:41.984205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4044e3-d77a-4adc-be31-344d9ad123c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:41.984474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4044e3-d77a-4adc-be31-344d9ad123c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:41.994553 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4044e3-d77a-4adc-be31-344d9ad123c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:41.995988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4044e3-d77a-4adc-be31-344d9ad123c3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-7f4044e3-d77a-4adc-be31-344d9ad123c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:42.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:42.011271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:42.012150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:42.012150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:42.012150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:42.012150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 293/583] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:42.012480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4044e3-d77a-4adc-be31-344d9ad123c3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:42.023460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd254a-2216-400e-a6c2-793833e5fdb5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:42.056614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d9715-f497-414e-9d5a-77d2743e3364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:42.057249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d9715-f497-414e-9d5a-77d2743e3364 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:42.057523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d9715-f497-414e-9d5a-77d2743e3364 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:42.057793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d9715-f497-414e-9d5a-77d2743e3364 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:42.058477 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d9715-f497-414e-9d5a-77d2743e3364 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:42.058778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d9715-f497-414e-9d5a-77d2743e3364 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:42.060199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d9715-f497-414e-9d5a-77d2743e3364 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:42.073989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 291/584] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:42.074871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d9715-f497-414e-9d5a-77d2743e3364 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:42.085186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:42.115400 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4044e3-d77a-4adc-be31-344d9ad123c3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:42.148301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946895c4-981b-4cc9-8da3-07d2014c0da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:42.148862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:42.149005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946895c4-981b-4cc9-8da3-07d2014c0da1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:42.149246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946895c4-981b-4cc9-8da3-07d2014c0da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:42.149521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946895c4-981b-4cc9-8da3-07d2014c0da1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:42.161420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946895c4-981b-4cc9-8da3-07d2014c0da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:42.162885 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946895c4-981b-4cc9-8da3-07d2014c0da1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:42.173418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d9715-f497-414e-9d5a-77d2743e3364 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-946895c4-981b-4cc9-8da3-07d2014c0da1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:42.177390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:42.178557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:42.179554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:42.179554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:42.179554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:42.183344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 294/585] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:42.184062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946895c4-981b-4cc9-8da3-07d2014c0da1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:42.206850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef352c-8c1d-4fe3-bab7-de8e117e660c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:42.207509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef352c-8c1d-4fe3-bab7-de8e117e660c None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:42.207785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef352c-8c1d-4fe3-bab7-de8e117e660c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:42.208053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef352c-8c1d-4fe3-bab7-de8e117e660c None admin] PATH_INFO: `/v3/projects/58466195aca94b188d82c1b3a647e767/users/d3272d57d5d24e5a990a6d7c86e6d4bc/roles/3eccbe0521ef44d88574b9acd9f988db` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:42.222079 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef352c-8c1d-4fe3-bab7-de8e117e660c None admin] RBAC: Authorizing `identity:create_grant(project_id=58466195aca94b188d82c1b3a647e767, user_id=d3272d57d5d24e5a990a6d7c86e6d4bc, role_id=3eccbe0521ef44d88574b9acd9f988db)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:42.223509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef352c-8c1d-4fe3-bab7-de8e117e660c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:42.235737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 292/586] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:15:42 2022] PUT /identity/v3/projects/58466195aca94b188d82c1b3a647e767/users/d3272d57d5d24e5a990a6d7c86e6d4bc/roles/3eccbe0521ef44d88574b9acd9f988db => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:15:42.236586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef352c-8c1d-4fe3-bab7-de8e117e660c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:42.287663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946895c4-981b-4cc9-8da3-07d2014c0da1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:42.314638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:42.327568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a26f5-22a3-404f-b34c-04a77792a141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:42.328288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a26f5-22a3-404f-b34c-04a77792a141 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:42.328466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a26f5-22a3-404f-b34c-04a77792a141 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:42.328739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a26f5-22a3-404f-b34c-04a77792a141 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:42.329437 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a26f5-22a3-404f-b34c-04a77792a141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:42.329740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a26f5-22a3-404f-b34c-04a77792a141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:42.331271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a26f5-22a3-404f-b34c-04a77792a141 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:42.344777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 295/587] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:42.346119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a26f5-22a3-404f-b34c-04a77792a141 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:42.359847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef352c-8c1d-4fe3-bab7-de8e117e660c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:42.386719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:42.399330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6ab01-2866-467d-a0c5-11e5a99431b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:42.400098 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6ab01-2866-467d-a0c5-11e5a99431b9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:42.400413 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6ab01-2866-467d-a0c5-11e5a99431b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:42.400689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6ab01-2866-467d-a0c5-11e5a99431b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:42.401384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6ab01-2866-467d-a0c5-11e5a99431b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:42.401690 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6ab01-2866-467d-a0c5-11e5a99431b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:42.403120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6ab01-2866-467d-a0c5-11e5a99431b9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:42.419365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 293/588] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:42.420765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6ab01-2866-467d-a0c5-11e5a99431b9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:42.464000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a26f5-22a3-404f-b34c-04a77792a141 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:42.500389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d8905-e7ca-4520-bd19-4d94bb2dcc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:42.500996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d8905-e7ca-4520-bd19-4d94bb2dcc6b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:42.501306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d8905-e7ca-4520-bd19-4d94bb2dcc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:42.501573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d8905-e7ca-4520-bd19-4d94bb2dcc6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:42.502359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d8905-e7ca-4520-bd19-4d94bb2dcc6b None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=7aac7cd1742449328facf2902483f259) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:42.502661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d8905-e7ca-4520-bd19-4d94bb2dcc6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:42.504122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d8905-e7ca-4520-bd19-4d94bb2dcc6b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:42.517108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 296/589] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:42 2022] GET /identity/v3/role_assignments?user.id=7aac7cd1742449328facf2902483f259&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:42.517108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d8905-e7ca-4520-bd19-4d94bb2dcc6b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:42.539161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6ab01-2866-467d-a0c5-11e5a99431b9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:42.578291 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:42.580251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5f5fa-ce74-4617-8e9d-d736c4b35afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:42.580886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5f5fa-ce74-4617-8e9d-d736c4b35afa None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:42.581209 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5f5fa-ce74-4617-8e9d-d736c4b35afa None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:42.581485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5f5fa-ce74-4617-8e9d-d736c4b35afa None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:42.582365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5f5fa-ce74-4617-8e9d-d736c4b35afa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:42.582678 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5f5fa-ce74-4617-8e9d-d736c4b35afa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:42.584090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5f5fa-ce74-4617-8e9d-d736c4b35afa None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:42.603279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 294/590] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:42.603279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5f5fa-ce74-4617-8e9d-d736c4b35afa None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:42.648396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d8905-e7ca-4520-bd19-4d94bb2dcc6b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:42.655189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:42.687054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba3437-f712-4ef9-87a4-0ffd2a05bddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:42.687834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba3437-f712-4ef9-87a4-0ffd2a05bddf None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:42.688211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba3437-f712-4ef9-87a4-0ffd2a05bddf None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:42.688584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba3437-f712-4ef9-87a4-0ffd2a05bddf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:42.697236 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba3437-f712-4ef9-87a4-0ffd2a05bddf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:42.698898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba3437-f712-4ef9-87a4-0ffd2a05bddf None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8aba3437-f712-4ef9-87a4-0ffd2a05bddf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:42.711873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:42.713030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:42.714020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:42.714020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:42.714020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:42.714020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:42.714020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:42.714020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:42.714020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:42.714020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:42.714020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 04:15:42.714020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:42.714579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 297/591] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:42.719517 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba3437-f712-4ef9-87a4-0ffd2a05bddf None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:42.731156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5f5fa-ce74-4617-8e9d-d736c4b35afa None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:42.774334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8cc91-589b-4dfa-a77d-676d294d5a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:42.775154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8cc91-589b-4dfa-a77d-676d294d5a17 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:42.775500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8cc91-589b-4dfa-a77d-676d294d5a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:42.775839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8cc91-589b-4dfa-a77d-676d294d5a17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:42.783189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8cc91-589b-4dfa-a77d-676d294d5a17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:42.784710 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8cc91-589b-4dfa-a77d-676d294d5a17 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d1a8cc91-589b-4dfa-a77d-676d294d5a17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:42.796627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:42.797594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:42.798478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:42.798478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:42.798478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:42.798478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:42.798478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:42.798478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:42.798478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:42.798478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:42.798478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 295/592] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:42.798985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8cc91-589b-4dfa-a77d-676d294d5a17 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:42.849192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba3437-f712-4ef9-87a4-0ffd2a05bddf None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:42.860043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:42.887521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90ac0f-fc77-4192-b6af-8934185295ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:42.888266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90ac0f-fc77-4192-b6af-8934185295ef None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:42.888612 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90ac0f-fc77-4192-b6af-8934185295ef None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:42.888954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90ac0f-fc77-4192-b6af-8934185295ef None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:42.897258 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90ac0f-fc77-4192-b6af-8934185295ef None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:42.898978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90ac0f-fc77-4192-b6af-8934185295ef None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0b90ac0f-fc77-4192-b6af-8934185295ef None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:42.910926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:42.912804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:42.914746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:42.914746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:42.914746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:42.914746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:42.914746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:42.914746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:42.914746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:42.914746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:42.914746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 04:15:42.914746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:42.914746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 298/593] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:42.914746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90ac0f-fc77-4192-b6af-8934185295ef None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:42.920440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8cc91-589b-4dfa-a77d-676d294d5a17 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:42.922773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:42.964372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cab0f0-d1f4-41cd-9520-3d6e6fc8eaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:42.965003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cab0f0-d1f4-41cd-9520-3d6e6fc8eaf7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:42.965412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cab0f0-d1f4-41cd-9520-3d6e6fc8eaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:42.965685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cab0f0-d1f4-41cd-9520-3d6e6fc8eaf7 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:42.966414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cab0f0-d1f4-41cd-9520-3d6e6fc8eaf7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:42.966719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cab0f0-d1f4-41cd-9520-3d6e6fc8eaf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:42.968258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cab0f0-d1f4-41cd-9520-3d6e6fc8eaf7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:42.987506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 296/594] 10.222.0.60 () {56 vars in 1137 bytes} [Sat Sep 17 04:15:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:42.988472 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cab0f0-d1f4-41cd-9520-3d6e6fc8eaf7 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:43.025703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90ac0f-fc77-4192-b6af-8934185295ef None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:43.062139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a4740-cc93-4ac5-9eb0-05a1d8ceb3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:43.062754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a4740-cc93-4ac5-9eb0-05a1d8ceb3be None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:43.063024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a4740-cc93-4ac5-9eb0-05a1d8ceb3be None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:43.063289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a4740-cc93-4ac5-9eb0-05a1d8ceb3be None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:43.064048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a4740-cc93-4ac5-9eb0-05a1d8ceb3be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:43.064392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a4740-cc93-4ac5-9eb0-05a1d8ceb3be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:43.065895 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a4740-cc93-4ac5-9eb0-05a1d8ceb3be None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:43.078534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 299/595] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:43.079430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a4740-cc93-4ac5-9eb0-05a1d8ceb3be None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:43.100621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cab0f0-d1f4-41cd-9520-3d6e6fc8eaf7 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:43.108214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:43.139103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21596e69-fbf1-4e54-9d0d-07c23706375d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:43.139720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21596e69-fbf1-4e54-9d0d-07c23706375d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:43.139997 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21596e69-fbf1-4e54-9d0d-07c23706375d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:43.140375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21596e69-fbf1-4e54-9d0d-07c23706375d None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:43.141238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21596e69-fbf1-4e54-9d0d-07c23706375d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:43.141545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21596e69-fbf1-4e54-9d0d-07c23706375d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:43.143123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21596e69-fbf1-4e54-9d0d-07c23706375d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:43.164286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 297/596] 10.222.0.60 () {56 vars in 1139 bytes} [Sat Sep 17 04:15:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:43.165105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21596e69-fbf1-4e54-9d0d-07c23706375d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:43.185933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a4740-cc93-4ac5-9eb0-05a1d8ceb3be None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:43.188703 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:43.221895 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ff7a2-c81d-499a-b5e1-fa68bf45d2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:43.222575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ff7a2-c81d-499a-b5e1-fa68bf45d2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:43.222846 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ff7a2-c81d-499a-b5e1-fa68bf45d2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:43.223157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ff7a2-c81d-499a-b5e1-fa68bf45d2a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:43.233753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ff7a2-c81d-499a-b5e1-fa68bf45d2a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:43.235223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ff7a2-c81d-499a-b5e1-fa68bf45d2a6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c25ff7a2-c81d-499a-b5e1-fa68bf45d2a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:43.250774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:43.253255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:43.255318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:43.255318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:43.255318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:43.255318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 300/597] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:43.255318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ff7a2-c81d-499a-b5e1-fa68bf45d2a6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:43.273771 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21596e69-fbf1-4e54-9d0d-07c23706375d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:43.310015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e93b6-f400-4108-8b08-f4f33de6c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:43.310922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e93b6-f400-4108-8b08-f4f33de6c4f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:43.311202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e93b6-f400-4108-8b08-f4f33de6c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:43.311473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e93b6-f400-4108-8b08-f4f33de6c4f2 None admin] PATH_INFO: `/v3/projects/58466195aca94b188d82c1b3a647e767/users/acf8a0ae03c94aa78cc2ab0e4e1c87b6/roles/3eccbe0521ef44d88574b9acd9f988db` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:43.328155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e93b6-f400-4108-8b08-f4f33de6c4f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=58466195aca94b188d82c1b3a647e767, user_id=acf8a0ae03c94aa78cc2ab0e4e1c87b6, role_id=3eccbe0521ef44d88574b9acd9f988db)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:43.329640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e93b6-f400-4108-8b08-f4f33de6c4f2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:43.339545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:43.353401 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 298/598] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:15:43 2022] PUT /identity/v3/projects/58466195aca94b188d82c1b3a647e767/users/acf8a0ae03c94aa78cc2ab0e4e1c87b6/roles/3eccbe0521ef44d88574b9acd9f988db => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:15:43.354448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e93b6-f400-4108-8b08-f4f33de6c4f2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:43.362212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ff7a2-c81d-499a-b5e1-fa68bf45d2a6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:43.409637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532c608-f2b1-4a3b-86a7-d19775609842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:43.410361 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532c608-f2b1-4a3b-86a7-d19775609842 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:43.410634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532c608-f2b1-4a3b-86a7-d19775609842 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:43.410902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532c608-f2b1-4a3b-86a7-d19775609842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:43.436272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532c608-f2b1-4a3b-86a7-d19775609842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:43.437730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532c608-f2b1-4a3b-86a7-d19775609842 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a532c608-f2b1-4a3b-86a7-d19775609842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:43.470235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:43.472481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:43.474613 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:43.474613 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:43.474613 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:43.474613 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 301/599] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:43.480839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532c608-f2b1-4a3b-86a7-d19775609842 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:43.501980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:43.589873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532c608-f2b1-4a3b-86a7-d19775609842 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:43.595335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e93b6-f400-4108-8b08-f4f33de6c4f2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:43.627362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:43.633930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ccf14-fb51-41c2-8bfc-775ea51353fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:43.634619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ccf14-fb51-41c2-8bfc-775ea51353fc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:43.634896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ccf14-fb51-41c2-8bfc-775ea51353fc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:43.635290 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ccf14-fb51-41c2-8bfc-775ea51353fc None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:43.636100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ccf14-fb51-41c2-8bfc-775ea51353fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:43.636396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ccf14-fb51-41c2-8bfc-775ea51353fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:43.638105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ccf14-fb51-41c2-8bfc-775ea51353fc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:43.641856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a893d91-6e3f-4f69-a991-c8b9e9245119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:43.642484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a893d91-6e3f-4f69-a991-c8b9e9245119 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:43.642762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a893d91-6e3f-4f69-a991-c8b9e9245119 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:43.643035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a893d91-6e3f-4f69-a991-c8b9e9245119 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:43.643688 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a893d91-6e3f-4f69-a991-c8b9e9245119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:43.643992 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a893d91-6e3f-4f69-a991-c8b9e9245119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:43.645397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a893d91-6e3f-4f69-a991-c8b9e9245119 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:43.652059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 302/600] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:43.657507 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ccf14-fb51-41c2-8bfc-775ea51353fc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:43.659330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 299/601] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:43.668035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a893d91-6e3f-4f69-a991-c8b9e9245119 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:43.767907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a893d91-6e3f-4f69-a991-c8b9e9245119 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:43.792224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ccf14-fb51-41c2-8bfc-775ea51353fc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:43.831983 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d999a-93c1-4672-81dc-a5cdc9acfc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:43.832626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d999a-93c1-4672-81dc-a5cdc9acfc94 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:43.832906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d999a-93c1-4672-81dc-a5cdc9acfc94 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:43.833203 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d999a-93c1-4672-81dc-a5cdc9acfc94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:43.841453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:43.847809 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d999a-93c1-4672-81dc-a5cdc9acfc94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:43.849307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d999a-93c1-4672-81dc-a5cdc9acfc94 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:43.851151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99edc8-724c-4cb1-afda-5b8eaae6f04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:43.851803 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99edc8-724c-4cb1-afda-5b8eaae6f04f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:43.852077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99edc8-724c-4cb1-afda-5b8eaae6f04f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:43.852341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99edc8-724c-4cb1-afda-5b8eaae6f04f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:43.865521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99edc8-724c-4cb1-afda-5b8eaae6f04f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:43.867353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99edc8-724c-4cb1-afda-5b8eaae6f04f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a06d999a-93c1-4672-81dc-a5cdc9acfc94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:43.873781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:43.874673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:43.875557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:43.875557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:43.875557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:43.875557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:43.875557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:43.875557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:43.875557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:43.875557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:43.875977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 303/602] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:43.885405 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d999a-93c1-4672-81dc-a5cdc9acfc94 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-5d99edc8-724c-4cb1-afda-5b8eaae6f04f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:43.889702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:43.891418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:43.892517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:43.892517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:43.892517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:43.892517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:43.892517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:43.892517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:43.892517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:43.892517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:43.892517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 300/603] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:43.906183 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99edc8-724c-4cb1-afda-5b8eaae6f04f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:43.947600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:44.044275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d999a-93c1-4672-81dc-a5cdc9acfc94 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:44.104981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99edc8-724c-4cb1-afda-5b8eaae6f04f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:44.108825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7baff5d-6347-40bf-b51e-a4221789b560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:44.109524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7baff5d-6347-40bf-b51e-a4221789b560 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:44.109800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7baff5d-6347-40bf-b51e-a4221789b560 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:44.110075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7baff5d-6347-40bf-b51e-a4221789b560 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:44.110777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7baff5d-6347-40bf-b51e-a4221789b560 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:44.111086 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7baff5d-6347-40bf-b51e-a4221789b560 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:44.112544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7baff5d-6347-40bf-b51e-a4221789b560 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:44.137461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 304/604] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:44.143240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7baff5d-6347-40bf-b51e-a4221789b560 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:44.149943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f11e3-c589-4d97-a687-928c9e3445d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:44.150578 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f11e3-c589-4d97-a687-928c9e3445d9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:44.150857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f11e3-c589-4d97-a687-928c9e3445d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:44.151124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f11e3-c589-4d97-a687-928c9e3445d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:44.151826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f11e3-c589-4d97-a687-928c9e3445d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:44.152133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f11e3-c589-4d97-a687-928c9e3445d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:44.153771 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f11e3-c589-4d97-a687-928c9e3445d9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:44.183252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 301/605] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:44.190341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f11e3-c589-4d97-a687-928c9e3445d9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:44.248013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7baff5d-6347-40bf-b51e-a4221789b560 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:44.262229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:44.277854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:44.285789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc651f-4306-4b82-ac99-1374469da18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:44.286467 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc651f-4306-4b82-ac99-1374469da18b None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:44.286817 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc651f-4306-4b82-ac99-1374469da18b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:44.287094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc651f-4306-4b82-ac99-1374469da18b None admin] PATH_INFO: `/v3/projects/58466195aca94b188d82c1b3a647e767/users/c9360be7ee9e4739b20c59955d28da47/roles/3eccbe0521ef44d88574b9acd9f988db` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:44.303585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc651f-4306-4b82-ac99-1374469da18b None admin] RBAC: Authorizing `identity:create_grant(project_id=58466195aca94b188d82c1b3a647e767, user_id=c9360be7ee9e4739b20c59955d28da47, role_id=3eccbe0521ef44d88574b9acd9f988db)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:44.304965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc651f-4306-4b82-ac99-1374469da18b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:44.338780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 305/606] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:15:44 2022] PUT /identity/v3/projects/58466195aca94b188d82c1b3a647e767/users/c9360be7ee9e4739b20c59955d28da47/roles/3eccbe0521ef44d88574b9acd9f988db => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:15:44.372516 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f11e3-c589-4d97-a687-928c9e3445d9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:44.426698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f13fc3-fe08-47b1-990e-0fccf51ad725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:44.427407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f13fc3-fe08-47b1-990e-0fccf51ad725 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:44.427679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f13fc3-fe08-47b1-990e-0fccf51ad725 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:44.427950 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f13fc3-fe08-47b1-990e-0fccf51ad725 None admin] PATH_INFO: `/v3/projects/58466195aca94b188d82c1b3a647e767/users/6adf758d5fb9430189ee9057b4b7ea97/roles/3eccbe0521ef44d88574b9acd9f988db` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:44.451820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f13fc3-fe08-47b1-990e-0fccf51ad725 None admin] RBAC: Authorizing `identity:create_grant(project_id=58466195aca94b188d82c1b3a647e767, user_id=6adf758d5fb9430189ee9057b4b7ea97, role_id=3eccbe0521ef44d88574b9acd9f988db)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:44.453338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f13fc3-fe08-47b1-990e-0fccf51ad725 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:44.473119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 302/607] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:15:44 2022] PUT /identity/v3/projects/58466195aca94b188d82c1b3a647e767/users/6adf758d5fb9430189ee9057b4b7ea97/roles/3eccbe0521ef44d88574b9acd9f988db => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:15:44.672896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3ea54-2e28-4e60-a2f7-76ffe33560e2 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:44.673453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3ea54-2e28-4e60-a2f7-76ffe33560e2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:44.673453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3ea54-2e28-4e60-a2f7-76ffe33560e2 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:44.675064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 306/608] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:44.680778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b03e09-c64a-42f7-b9d3-716216d3ccef None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:44.681443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b03e09-c64a-42f7-b9d3-716216d3ccef None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:44.681443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b03e09-c64a-42f7-b9d3-716216d3ccef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:44.706658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e9b03e09-c64a-42f7-b9d3-716216d3ccef None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:44.708034 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e9b03e09-c64a-42f7-b9d3-716216d3ccef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:44.796498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 303/609] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:44.827574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ab743-7864-46a9-9767-b307a73c0658 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:44.828191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ab743-7864-46a9-9767-b307a73c0658 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:44.828650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ab743-7864-46a9-9767-b307a73c0658 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:44.830491 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 307/610] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:44.838010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca59433-d247-4709-a210-5f0bbaede859 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:44.838291 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca59433-d247-4709-a210-5f0bbaede859 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:44.838561 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca59433-d247-4709-a210-5f0bbaede859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:44.861355 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0ca59433-d247-4709-a210-5f0bbaede859 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:44.862536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0ca59433-d247-4709-a210-5f0bbaede859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:44.896482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:44.954725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 304/611] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:45.077997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ab743-7864-46a9-9767-b307a73c0658 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:45.206121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ab743-7864-46a9-9767-b307a73c0658 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:45.259123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edab89f-c4b6-4afc-804d-16e9bcd766a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:45.259793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edab89f-c4b6-4afc-804d-16e9bcd766a1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:45.260067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edab89f-c4b6-4afc-804d-16e9bcd766a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:45.260335 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edab89f-c4b6-4afc-804d-16e9bcd766a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:45.274748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edab89f-c4b6-4afc-804d-16e9bcd766a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4edab89f-c4b6-4afc-804d-16e9bcd766a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:45.279554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:45.280483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:45.281328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 308/612] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:45.283573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca59433-d247-4709-a210-5f0bbaede859 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:45.291419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edab89f-c4b6-4afc-804d-16e9bcd766a1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:45.336249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:45.409599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca59433-d247-4709-a210-5f0bbaede859 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:45.445670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfabf7c-9bf7-4619-91e8-ec96db44f055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:45.446698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfabf7c-9bf7-4619-91e8-ec96db44f055 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:45.447250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfabf7c-9bf7-4619-91e8-ec96db44f055 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:45.447727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfabf7c-9bf7-4619-91e8-ec96db44f055 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:45.458651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfabf7c-9bf7-4619-91e8-ec96db44f055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-acfabf7c-9bf7-4619-91e8-ec96db44f055 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:45.463963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:45.464919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:45.465799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:45.465799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:45.465799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:45.465799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:45.465799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:45.465799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:45.465799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:45.465799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:45.465799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:45.465799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:45.465799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:45.465799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:45.468468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 305/613] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:45.476544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfabf7c-9bf7-4619-91e8-ec96db44f055 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:45.482830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edab89f-c4b6-4afc-804d-16e9bcd766a1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:45.501557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:45.563338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addaafc-6000-4a0e-8ab6-9c5115ee2f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:45.564070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addaafc-6000-4a0e-8ab6-9c5115ee2f69 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:45.564266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addaafc-6000-4a0e-8ab6-9c5115ee2f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:45.564519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addaafc-6000-4a0e-8ab6-9c5115ee2f69 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:45.565209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addaafc-6000-4a0e-8ab6-9c5115ee2f69 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:45.565503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addaafc-6000-4a0e-8ab6-9c5115ee2f69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:45.566998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addaafc-6000-4a0e-8ab6-9c5115ee2f69 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:45.583729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfabf7c-9bf7-4619-91e8-ec96db44f055 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:45.584810 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 309/614] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:45.590841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addaafc-6000-4a0e-8ab6-9c5115ee2f69 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:45.621749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866393b5-5561-4e6f-8cdb-8da705414105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:45.622773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866393b5-5561-4e6f-8cdb-8da705414105 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:45.623264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866393b5-5561-4e6f-8cdb-8da705414105 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:45.623760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866393b5-5561-4e6f-8cdb-8da705414105 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:45.624683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866393b5-5561-4e6f-8cdb-8da705414105 None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:45.625274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866393b5-5561-4e6f-8cdb-8da705414105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:45.626931 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866393b5-5561-4e6f-8cdb-8da705414105 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:45.636525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 306/615] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:45.643777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866393b5-5561-4e6f-8cdb-8da705414105 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:45.760629 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:45.789805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addaafc-6000-4a0e-8ab6-9c5115ee2f69 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:45.813659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:45.839696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866393b5-5561-4e6f-8cdb-8da705414105 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:45.856165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83707c0-2ad9-492b-8aba-3659dbb358a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:45.856820 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83707c0-2ad9-492b-8aba-3659dbb358a9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:45.857123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83707c0-2ad9-492b-8aba-3659dbb358a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:45.857410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83707c0-2ad9-492b-8aba-3659dbb358a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:45.879159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83707c0-2ad9-492b-8aba-3659dbb358a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:45.880632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83707c0-2ad9-492b-8aba-3659dbb358a9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e83707c0-2ad9-492b-8aba-3659dbb358a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:45.910333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:45.911836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:45.913850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:45.913850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:45.913850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:45.914429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed860b-2e92-4de6-ab7c-7464c91207b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:45.914429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed860b-2e92-4de6-ab7c-7464c91207b1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:45.914429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed860b-2e92-4de6-ab7c-7464c91207b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:45.914429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed860b-2e92-4de6-ab7c-7464c91207b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:45.914868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 310/616] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:45.916855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83707c0-2ad9-492b-8aba-3659dbb358a9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:45.936750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed860b-2e92-4de6-ab7c-7464c91207b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:45.938234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed860b-2e92-4de6-ab7c-7464c91207b1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d7ed860b-2e92-4de6-ab7c-7464c91207b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:45.957576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:45.959850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:45.962033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:45.962033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:45.962033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:45.962033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 307/617] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:45.968292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed860b-2e92-4de6-ab7c-7464c91207b1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:46.132822 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83707c0-2ad9-492b-8aba-3659dbb358a9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:46.141574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed860b-2e92-4de6-ab7c-7464c91207b1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:46.173687 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848f8f2-1c0e-40e5-9b62-b24b29c422b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:46.174729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848f8f2-1c0e-40e5-9b62-b24b29c422b6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:46.175327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848f8f2-1c0e-40e5-9b62-b24b29c422b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:46.175896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848f8f2-1c0e-40e5-9b62-b24b29c422b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:46.176733 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848f8f2-1c0e-40e5-9b62-b24b29c422b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:46.177286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848f8f2-1c0e-40e5-9b62-b24b29c422b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:46.178991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848f8f2-1c0e-40e5-9b62-b24b29c422b6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:46.182382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:46.184501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e577c4c-ab4c-4a87-9ba0-d3901f475787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:46.185134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e577c4c-ab4c-4a87-9ba0-d3901f475787 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:46.185493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e577c4c-ab4c-4a87-9ba0-d3901f475787 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:46.185915 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e577c4c-ab4c-4a87-9ba0-d3901f475787 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:46.186559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e577c4c-ab4c-4a87-9ba0-d3901f475787 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:46.186854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e577c4c-ab4c-4a87-9ba0-d3901f475787 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:46.188220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e577c4c-ab4c-4a87-9ba0-d3901f475787 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:46.189837 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:46.198912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 311/618] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:46.201332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d369c-0b79-460c-9805-79299595b6e6 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:46.202033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d369c-0b79-460c-9805-79299595b6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:46.202723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d369c-0b79-460c-9805-79299595b6e6 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:46.204446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 312/619] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:46.205406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d369c-0b79-460c-9805-79299595b6e6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:46.207695 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 308/620] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:46.213439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187c54b-d4a0-43a2-9ef7-d2d95f3b03aa None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:46.213760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187c54b-d4a0-43a2-9ef7-d2d95f3b03aa None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:46.214012 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187c54b-d4a0-43a2-9ef7-d2d95f3b03aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:46.232046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2187c54b-d4a0-43a2-9ef7-d2d95f3b03aa None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:46.233195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2187c54b-d4a0-43a2-9ef7-d2d95f3b03aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:46.302155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 309/621] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:46.303081 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187c54b-d4a0-43a2-9ef7-d2d95f3b03aa None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:46.327025 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d369c-0b79-460c-9805-79299595b6e6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:46.389801 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77eb30-eaf3-4920-9f07-39ca134cfc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:46.390792 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77eb30-eaf3-4920-9f07-39ca134cfc77 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:46.391286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77eb30-eaf3-4920-9f07-39ca134cfc77 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:46.391816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77eb30-eaf3-4920-9f07-39ca134cfc77 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:46.401018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187c54b-d4a0-43a2-9ef7-d2d95f3b03aa None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:46.401831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77eb30-eaf3-4920-9f07-39ca134cfc77 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:46.410570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77eb30-eaf3-4920-9f07-39ca134cfc77 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:46.430126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-cb77eb30-eaf3-4920-9f07-39ca134cfc77 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:46.433482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:46.434634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:46.435582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:46.435582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:46.435582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:46.435582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:46.435582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:46.435582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:46.435582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:46.435582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:46.435582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:15:46.435582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:46.437825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 313/622] 10.222.0.60 () {56 vars in 1093 bytes} [Sat Sep 17 04:15:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:46.443119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c326454-ace0-49b8-a9a0-6f72eb36396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:46.443616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77eb30-eaf3-4920-9f07-39ca134cfc77 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:46.443990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c326454-ace0-49b8-a9a0-6f72eb36396f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:46.444361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c326454-ace0-49b8-a9a0-6f72eb36396f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:46.444772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c326454-ace0-49b8-a9a0-6f72eb36396f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:46.457208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c326454-ace0-49b8-a9a0-6f72eb36396f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:46.457208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c326454-ace0-49b8-a9a0-6f72eb36396f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-0c326454-ace0-49b8-a9a0-6f72eb36396f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:46.475830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:46.476813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:46.477838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:46.477838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:46.477838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:46.477838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:46.477838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:46.477838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:46.477838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:46.477838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:46.477838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 04:15:46.477838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:46.484524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 310/623] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:15:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:46.489379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c326454-ace0-49b8-a9a0-6f72eb36396f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:46.563794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:46.610236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c326454-ace0-49b8-a9a0-6f72eb36396f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:46.652269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77eb30-eaf3-4920-9f07-39ca134cfc77 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:46.670658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895d82a-f500-427b-bea3-663632af24c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:46.671270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895d82a-f500-427b-bea3-663632af24c3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:46.671552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895d82a-f500-427b-bea3-663632af24c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:46.671822 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895d82a-f500-427b-bea3-663632af24c3 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:46.672600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895d82a-f500-427b-bea3-663632af24c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:46.672822 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895d82a-f500-427b-bea3-663632af24c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:46.674266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895d82a-f500-427b-bea3-663632af24c3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:46.702452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 311/624] 10.222.0.60 () {56 vars in 1135 bytes} [Sat Sep 17 04:15:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:46.703313 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895d82a-f500-427b-bea3-663632af24c3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:46.710363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b22af-db2d-4cb1-94dc-0e710e997bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:46.711341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b22af-db2d-4cb1-94dc-0e710e997bdd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:46.711839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b22af-db2d-4cb1-94dc-0e710e997bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:46.712306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b22af-db2d-4cb1-94dc-0e710e997bdd None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:46.713368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b22af-db2d-4cb1-94dc-0e710e997bdd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:46.713904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b22af-db2d-4cb1-94dc-0e710e997bdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:46.715535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b22af-db2d-4cb1-94dc-0e710e997bdd None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:46.735263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 314/625] 10.222.0.60 () {56 vars in 1133 bytes} [Sat Sep 17 04:15:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:46.736470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b22af-db2d-4cb1-94dc-0e710e997bdd None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:46.755562 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:46.811887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895d82a-f500-427b-bea3-663632af24c3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:46.845170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6420ddb-fc76-45c9-af5d-407b32cda94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:46.845848 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6420ddb-fc76-45c9-af5d-407b32cda94e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:46.846073 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6420ddb-fc76-45c9-af5d-407b32cda94e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:46.846318 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6420ddb-fc76-45c9-af5d-407b32cda94e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:46.849514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b22af-db2d-4cb1-94dc-0e710e997bdd None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:46.857386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6420ddb-fc76-45c9-af5d-407b32cda94e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c6420ddb-fc76-45c9-af5d-407b32cda94e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:46.862330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:46.863379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:46.864259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 312/626] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:46.864975 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6420ddb-fc76-45c9-af5d-407b32cda94e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:46.876375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:46.907607 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8da519-4407-4237-8ec1-64d55111a0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:46.908783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8da519-4407-4237-8ec1-64d55111a0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:46.909233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8da519-4407-4237-8ec1-64d55111a0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:46.909657 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8da519-4407-4237-8ec1-64d55111a0d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:46.922934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8da519-4407-4237-8ec1-64d55111a0d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:46.925278 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8da519-4407-4237-8ec1-64d55111a0d5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-9b8da519-4407-4237-8ec1-64d55111a0d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:46.944305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:46.945409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:46.946368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:46.946368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:46.946368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:46.953336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 315/627] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:46.955252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8da519-4407-4237-8ec1-64d55111a0d5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:46.987093 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6420ddb-fc76-45c9-af5d-407b32cda94e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:46.988850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:47.023756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657a492-ac3e-4fec-b29e-622901c7721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:47.024371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657a492-ac3e-4fec-b29e-622901c7721f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.024645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657a492-ac3e-4fec-b29e-622901c7721f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.024942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657a492-ac3e-4fec-b29e-622901c7721f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.064323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8da519-4407-4237-8ec1-64d55111a0d5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:47.099465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657a492-ac3e-4fec-b29e-622901c7721f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:47.100891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657a492-ac3e-4fec-b29e-622901c7721f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:47.101010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d69b4d-e81b-4352-bafa-b182cd26893e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:47.101612 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d69b4d-e81b-4352-bafa-b182cd26893e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.101887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d69b4d-e81b-4352-bafa-b182cd26893e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.102157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d69b4d-e81b-4352-bafa-b182cd26893e None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.102818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d69b4d-e81b-4352-bafa-b182cd26893e None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:47.103180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d69b4d-e81b-4352-bafa-b182cd26893e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:47.104561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d69b4d-e81b-4352-bafa-b182cd26893e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:47.117812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 316/628] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:47.117812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d69b4d-e81b-4352-bafa-b182cd26893e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:47.145242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:47.204755 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 313/629] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:47.205956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98c290-b019-4f1a-b6c9-085866f7e8e1 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.206246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98c290-b019-4f1a-b6c9-085866f7e8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.206509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98c290-b019-4f1a-b6c9-085866f7e8e1 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.209443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 314/630] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:47.209443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98c290-b019-4f1a-b6c9-085866f7e8e1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:47.211568 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:47.217721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d69b4d-e81b-4352-bafa-b182cd26893e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:47.251293 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacf3f8-2815-4fbd-94dd-f8902fa07e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:47.251893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacf3f8-2815-4fbd-94dd-f8902fa07e25 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.252164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacf3f8-2815-4fbd-94dd-f8902fa07e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.252429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacf3f8-2815-4fbd-94dd-f8902fa07e25 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.253066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eacf3f8-2815-4fbd-94dd-f8902fa07e25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:47.253388 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eacf3f8-2815-4fbd-94dd-f8902fa07e25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:47.254726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eacf3f8-2815-4fbd-94dd-f8902fa07e25 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:47.266682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 317/631] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:47.267848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc391d-c532-46c8-bb43-734c22f2111e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.268131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc391d-c532-46c8-bb43-734c22f2111e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.268397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc391d-c532-46c8-bb43-734c22f2111e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.269766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 318/632] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:47.270298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc391d-c532-46c8-bb43-734c22f2111e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:47.308563 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98c290-b019-4f1a-b6c9-085866f7e8e1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:47.342559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d674e5d-3123-4715-affd-8f5103065985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:47.343184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d674e5d-3123-4715-affd-8f5103065985 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.343458 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d674e5d-3123-4715-affd-8f5103065985 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.343724 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d674e5d-3123-4715-affd-8f5103065985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.353474 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d674e5d-3123-4715-affd-8f5103065985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:47.354898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d674e5d-3123-4715-affd-8f5103065985 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:47.358058 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-2d674e5d-3123-4715-affd-8f5103065985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:47.369484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:47.370605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:47.371679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:47.371679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:47.371679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:47.371679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 315/633] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:47.371679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c1c2f-97e5-4270-ada7-24cb8bde6da8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.372016 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c1c2f-97e5-4270-ada7-24cb8bde6da8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.372212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c1c2f-97e5-4270-ada7-24cb8bde6da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.375314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc391d-c532-46c8-bb43-734c22f2111e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:47.389412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-159c1c2f-97e5-4270-ada7-24cb8bde6da8 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:47.390580 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-159c1c2f-97e5-4270-ada7-24cb8bde6da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:47.410913 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955b927-d690-4bfd-b473-2ae4e8ee0287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:47.411540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955b927-d690-4bfd-b473-2ae4e8ee0287 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.411812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955b927-d690-4bfd-b473-2ae4e8ee0287 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.412079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955b927-d690-4bfd-b473-2ae4e8ee0287 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.421804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955b927-d690-4bfd-b473-2ae4e8ee0287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:47.423233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955b927-d690-4bfd-b473-2ae4e8ee0287 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8955b927-d690-4bfd-b473-2ae4e8ee0287 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:47.437606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:47.438573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:47.439479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:47.439479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:47.439479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:47.439479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 319/634] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:47.439713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:47.439787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7ef6f-6728-451c-ad63-27df9f2277b0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.440059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7ef6f-6728-451c-ad63-27df9f2277b0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.440330 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7ef6f-6728-451c-ad63-27df9f2277b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.457603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6af7ef6f-6728-451c-ad63-27df9f2277b0 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:47.458708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6af7ef6f-6728-451c-ad63-27df9f2277b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:47.459029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 316/635] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:47.459847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c1c2f-97e5-4270-ada7-24cb8bde6da8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:47.533817 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 320/636] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:47.535041 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7ef6f-6728-451c-ad63-27df9f2277b0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:47.577414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c1c2f-97e5-4270-ada7-24cb8bde6da8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:47.618731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:47.620036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13443ee0-e034-4e64-9f83-8719686ba24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:47.620857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13443ee0-e034-4e64-9f83-8719686ba24d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.622067 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13443ee0-e034-4e64-9f83-8719686ba24d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.622343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13443ee0-e034-4e64-9f83-8719686ba24d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.631150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13443ee0-e034-4e64-9f83-8719686ba24d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:47.632643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13443ee0-e034-4e64-9f83-8719686ba24d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-13443ee0-e034-4e64-9f83-8719686ba24d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:47.647233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:47.648135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:47.649054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:47.649054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:47.649054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:47.649054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:47.649054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:47.649054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:47.649054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:47.649054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:47.650494 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 317/637] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:47.651489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13443ee0-e034-4e64-9f83-8719686ba24d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:47.684240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7ef6f-6728-451c-ad63-27df9f2277b0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:47.704022 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:47.731147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e72048-4048-48b2-99d4-a8960ff91034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:47.731819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e72048-4048-48b2-99d4-a8960ff91034 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.732184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e72048-4048-48b2-99d4-a8960ff91034 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.732542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e72048-4048-48b2-99d4-a8960ff91034 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.733315 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e72048-4048-48b2-99d4-a8960ff91034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:47.733706 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e72048-4048-48b2-99d4-a8960ff91034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:47.735183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e72048-4048-48b2-99d4-a8960ff91034 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:47.750850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 321/638] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:47.751871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e72048-4048-48b2-99d4-a8960ff91034 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:47.763400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13443ee0-e034-4e64-9f83-8719686ba24d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:47.800704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2a773-6226-4320-badf-3d163709b164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:47.801558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2a773-6226-4320-badf-3d163709b164 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.802089 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2a773-6226-4320-badf-3d163709b164 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.802357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2a773-6226-4320-badf-3d163709b164 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.802997 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2a773-6226-4320-badf-3d163709b164 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:47.803295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2a773-6226-4320-badf-3d163709b164 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:47.804662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2a773-6226-4320-badf-3d163709b164 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:47.816426 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 318/639] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:47.817261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2a773-6226-4320-badf-3d163709b164 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:47.856600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e72048-4048-48b2-99d4-a8960ff91034 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:47.885680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:47.892472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ffd55-c5a3-4ed8-a2ea-8e89c500138e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:47.893179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ffd55-c5a3-4ed8-a2ea-8e89c500138e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.893451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ffd55-c5a3-4ed8-a2ea-8e89c500138e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.893715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ffd55-c5a3-4ed8-a2ea-8e89c500138e None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.894403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ffd55-c5a3-4ed8-a2ea-8e89c500138e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:47.894698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ffd55-c5a3-4ed8-a2ea-8e89c500138e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:47.896112 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ffd55-c5a3-4ed8-a2ea-8e89c500138e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:47.907790 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 322/640] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:47.908574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ffd55-c5a3-4ed8-a2ea-8e89c500138e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:47.916749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2a773-6226-4320-badf-3d163709b164 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:47.919419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:47.953577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8abf73-dfb9-4761-aa21-64e93600a3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:47.954211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8abf73-dfb9-4761-aa21-64e93600a3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:47.954484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8abf73-dfb9-4761-aa21-64e93600a3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:47.954756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8abf73-dfb9-4761-aa21-64e93600a3f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:47.964653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8abf73-dfb9-4761-aa21-64e93600a3f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-dc8abf73-dfb9-4761-aa21-64e93600a3f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:47.969534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:47.970658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:47.971605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 319/641] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:47.972643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8abf73-dfb9-4761-aa21-64e93600a3f5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:48.019352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ffd55-c5a3-4ed8-a2ea-8e89c500138e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:48.054009 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506cb49-cf40-41e8-b777-396505d3f027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:48.054620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506cb49-cf40-41e8-b777-396505d3f027 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:48.054890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506cb49-cf40-41e8-b777-396505d3f027 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:48.055155 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506cb49-cf40-41e8-b777-396505d3f027 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:48.062159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506cb49-cf40-41e8-b777-396505d3f027 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:48.063593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506cb49-cf40-41e8-b777-396505d3f027 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b506cb49-cf40-41e8-b777-396505d3f027 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:48.074944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:48.075949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:48.076917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:48.076917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:48.076917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:48.076917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:48.076917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:48.076917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:48.076917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:48.076917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:48.076917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 17 04:15:48.076917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:48.076917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 323/642] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:48.078077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506cb49-cf40-41e8-b777-396505d3f027 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:48.078149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8abf73-dfb9-4761-aa21-64e93600a3f5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:48.111535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f345cc-ece1-4d50-977a-7ff35c807e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:48.112155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f345cc-ece1-4d50-977a-7ff35c807e02 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:48.112425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f345cc-ece1-4d50-977a-7ff35c807e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:48.112708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f345cc-ece1-4d50-977a-7ff35c807e02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:48.123250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f345cc-ece1-4d50-977a-7ff35c807e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:48.126367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c6f345cc-ece1-4d50-977a-7ff35c807e02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:48.128225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:48.129270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:48.130181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 320/643] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:48.130996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f345cc-ece1-4d50-977a-7ff35c807e02 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:48.155404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:48.179557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506cb49-cf40-41e8-b777-396505d3f027 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:48.213095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c557593-51f0-4c61-9843-e9e1a1beb5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:48.213807 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c557593-51f0-4c61-9843-e9e1a1beb5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:48.214087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c557593-51f0-4c61-9843-e9e1a1beb5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:48.214357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c557593-51f0-4c61-9843-e9e1a1beb5c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:48.220944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c557593-51f0-4c61-9843-e9e1a1beb5c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:48.222424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c557593-51f0-4c61-9843-e9e1a1beb5c2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:48.228499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f345cc-ece1-4d50-977a-7ff35c807e02 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6c557593-51f0-4c61-9843-e9e1a1beb5c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:48.234042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:48.234990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:48.235892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:48.235892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:48.235892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:48.235892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:48.235892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:48.235892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:48.235892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:48.235892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:48.235892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 324/644] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:48.235892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c557593-51f0-4c61-9843-e9e1a1beb5c2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:48.263677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a6f1f-fcc3-4794-8ed4-0e261fb99cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:48.264336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a6f1f-fcc3-4794-8ed4-0e261fb99cdb None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:48.264610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a6f1f-fcc3-4794-8ed4-0e261fb99cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:48.264881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a6f1f-fcc3-4794-8ed4-0e261fb99cdb None admin] PATH_INFO: `/v3/projects/58466195aca94b188d82c1b3a647e767/users/7aac7cd1742449328facf2902483f259/roles/3eccbe0521ef44d88574b9acd9f988db` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:48.279081 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a6f1f-fcc3-4794-8ed4-0e261fb99cdb None admin] RBAC: Authorizing `identity:create_grant(project_id=58466195aca94b188d82c1b3a647e767, user_id=7aac7cd1742449328facf2902483f259, role_id=3eccbe0521ef44d88574b9acd9f988db)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:48.280459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a6f1f-fcc3-4794-8ed4-0e261fb99cdb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:48.294358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 321/645] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:15:48 2022] PUT /identity/v3/projects/58466195aca94b188d82c1b3a647e767/users/7aac7cd1742449328facf2902483f259/roles/3eccbe0521ef44d88574b9acd9f988db => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:15:48.295160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a6f1f-fcc3-4794-8ed4-0e261fb99cdb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:48.361840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c557593-51f0-4c61-9843-e9e1a1beb5c2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:48.375285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:48.391473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:48.408160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea545f-c77f-4384-a6b3-d98b29ff9176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:48.408858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea545f-c77f-4384-a6b3-d98b29ff9176 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:48.409191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea545f-c77f-4384-a6b3-d98b29ff9176 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:48.409454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea545f-c77f-4384-a6b3-d98b29ff9176 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:48.410116 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea545f-c77f-4384-a6b3-d98b29ff9176 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:48.410419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea545f-c77f-4384-a6b3-d98b29ff9176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:48.412038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea545f-c77f-4384-a6b3-d98b29ff9176 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:48.420510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a6f1f-fcc3-4794-8ed4-0e261fb99cdb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:48.426241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 325/646] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:48.426241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea545f-c77f-4384-a6b3-d98b29ff9176 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:48.463265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0804c-7963-4fae-8455-6c21651f05a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:48.463887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0804c-7963-4fae-8455-6c21651f05a0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:48.464170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0804c-7963-4fae-8455-6c21651f05a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:48.464431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0804c-7963-4fae-8455-6c21651f05a0 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:48.465196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0804c-7963-4fae-8455-6c21651f05a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:48.465548 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0804c-7963-4fae-8455-6c21651f05a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:48.467063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0804c-7963-4fae-8455-6c21651f05a0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:48.490651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 322/647] 10.222.0.60 () {56 vars in 1137 bytes} [Sat Sep 17 04:15:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:48.490854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0804c-7963-4fae-8455-6c21651f05a0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:48.537892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea545f-c77f-4384-a6b3-d98b29ff9176 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:48.571354 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755cca7-f05e-41c4-9fc6-4dc757037fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:48.572031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755cca7-f05e-41c4-9fc6-4dc757037fee None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:48.572311 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755cca7-f05e-41c4-9fc6-4dc757037fee None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:48.572580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755cca7-f05e-41c4-9fc6-4dc757037fee None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:48.573273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755cca7-f05e-41c4-9fc6-4dc757037fee None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:48.573571 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755cca7-f05e-41c4-9fc6-4dc757037fee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:48.574961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755cca7-f05e-41c4-9fc6-4dc757037fee None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:48.584212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 326/648] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:48.584984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755cca7-f05e-41c4-9fc6-4dc757037fee None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:48.598695 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0804c-7963-4fae-8455-6c21651f05a0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:48.620735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:48.630990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:48.636412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc21f9-6cf5-4b45-9992-7ea9e946de0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:48.637020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc21f9-6cf5-4b45-9992-7ea9e946de0d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:48.637344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc21f9-6cf5-4b45-9992-7ea9e946de0d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:48.637598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc21f9-6cf5-4b45-9992-7ea9e946de0d None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:48.638294 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc21f9-6cf5-4b45-9992-7ea9e946de0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:48.638597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc21f9-6cf5-4b45-9992-7ea9e946de0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:48.640046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc21f9-6cf5-4b45-9992-7ea9e946de0d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:48.651546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 323/649] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:48.652354 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc21f9-6cf5-4b45-9992-7ea9e946de0d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:48.715699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755cca7-f05e-41c4-9fc6-4dc757037fee None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:48.751271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910da7c-5c01-4ca6-ba58-209790559e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:48.751910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910da7c-5c01-4ca6-ba58-209790559e59 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:48.752186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910da7c-5c01-4ca6-ba58-209790559e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:48.752455 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910da7c-5c01-4ca6-ba58-209790559e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:48.763075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910da7c-5c01-4ca6-ba58-209790559e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:48.764526 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910da7c-5c01-4ca6-ba58-209790559e59 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:48.771351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc21f9-6cf5-4b45-9992-7ea9e946de0d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5910da7c-5c01-4ca6-ba58-209790559e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:48.778547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:48.779514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:48.785521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:48.785521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:48.785521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:48.785521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 327/650] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:48.785521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910da7c-5c01-4ca6-ba58-209790559e59 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:48.808332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194e59a-1fd4-4b8f-9ff0-d87305bbc33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:48.808979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194e59a-1fd4-4b8f-9ff0-d87305bbc33f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:48.809339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194e59a-1fd4-4b8f-9ff0-d87305bbc33f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:48.809708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194e59a-1fd4-4b8f-9ff0-d87305bbc33f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:48.867413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:48.871997 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:48.883051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194e59a-1fd4-4b8f-9ff0-d87305bbc33f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:48.884480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910da7c-5c01-4ca6-ba58-209790559e59 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:48.884594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194e59a-1fd4-4b8f-9ff0-d87305bbc33f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:48.925097 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aad7f4-70a3-47e6-8770-62783e908640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:48.925761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aad7f4-70a3-47e6-8770-62783e908640 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:48.926038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aad7f4-70a3-47e6-8770-62783e908640 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:48.926308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aad7f4-70a3-47e6-8770-62783e908640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:48.939987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aad7f4-70a3-47e6-8770-62783e908640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:48.942089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aad7f4-70a3-47e6-8770-62783e908640 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f5aad7f4-70a3-47e6-8770-62783e908640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:48.959893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:48.961069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:48.967343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:48.967343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:48.967343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:48.967343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 328/651] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:48.967343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aad7f4-70a3-47e6-8770-62783e908640 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:48.984556 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 324/652] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:48.986045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194e59a-1fd4-4b8f-9ff0-d87305bbc33f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:49.082062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:49.088949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aad7f4-70a3-47e6-8770-62783e908640 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:49.091552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194e59a-1fd4-4b8f-9ff0-d87305bbc33f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:49.121559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48db399-e567-445b-9263-0be70f81e6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:49.122168 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48db399-e567-445b-9263-0be70f81e6ce None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:49.122445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48db399-e567-445b-9263-0be70f81e6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:49.122716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48db399-e567-445b-9263-0be70f81e6ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:49.123439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48db399-e567-445b-9263-0be70f81e6ce None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=d3272d57d5d24e5a990a6d7c86e6d4bc) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:49.123744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48db399-e567-445b-9263-0be70f81e6ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:49.125196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48db399-e567-445b-9263-0be70f81e6ce None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:49.127997 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b675c-a599-4e8c-869a-94e58ca720b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:49.128612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b675c-a599-4e8c-869a-94e58ca720b2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:49.128886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b675c-a599-4e8c-869a-94e58ca720b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:49.129185 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b675c-a599-4e8c-869a-94e58ca720b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:49.129826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b675c-a599-4e8c-869a-94e58ca720b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:49.130126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b675c-a599-4e8c-869a-94e58ca720b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:49.136137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 329/653] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:48 2022] GET /identity/v3/role_assignments?user.id=d3272d57d5d24e5a990a6d7c86e6d4bc&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:49.136345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b675c-a599-4e8c-869a-94e58ca720b2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:49.136443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48db399-e567-445b-9263-0be70f81e6ce None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:49.136443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:49.158634 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 325/654] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:49.159757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b675c-a599-4e8c-869a-94e58ca720b2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:49.281345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b675c-a599-4e8c-869a-94e58ca720b2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:49.288757 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48db399-e567-445b-9263-0be70f81e6ce None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:49.320468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d5a9b-2d19-4b17-ba35-ab6345c2b8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:49.321304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d5a9b-2d19-4b17-ba35-ab6345c2b8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:49.321583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d5a9b-2d19-4b17-ba35-ab6345c2b8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:49.321924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d5a9b-2d19-4b17-ba35-ab6345c2b8f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:49.333663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d5a9b-2d19-4b17-ba35-ab6345c2b8f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:49.333795 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ef13f-fb54-4c65-a95d-23cdd46fe571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:49.334267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ef13f-fb54-4c65-a95d-23cdd46fe571 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:49.334545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ef13f-fb54-4c65-a95d-23cdd46fe571 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:49.334821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ef13f-fb54-4c65-a95d-23cdd46fe571 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:49.335048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d5a9b-2d19-4b17-ba35-ab6345c2b8f9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:49.335459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ef13f-fb54-4c65-a95d-23cdd46fe571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:49.335812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ef13f-fb54-4c65-a95d-23cdd46fe571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:49.337773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ef13f-fb54-4c65-a95d-23cdd46fe571 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:49.350804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 330/655] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-109d5a9b-2d19-4b17-ba35-ab6345c2b8f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:49.350983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:49.351862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:49.353570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:49.353570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:49.353570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:49.353739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ef13f-fb54-4c65-a95d-23cdd46fe571 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:49.353823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 326/656] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:49.357589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d5a9b-2d19-4b17-ba35-ab6345c2b8f9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:49.362383 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:49.432737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:49.482427 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ef13f-fb54-4c65-a95d-23cdd46fe571 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:49.486877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d5a9b-2d19-4b17-ba35-ab6345c2b8f9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:49.523712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cac2a-b7e5-4a98-bec4-2a0ebd133efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:49.524603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cac2a-b7e5-4a98-bec4-2a0ebd133efb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:49.525069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cac2a-b7e5-4a98-bec4-2a0ebd133efb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:49.525506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cac2a-b7e5-4a98-bec4-2a0ebd133efb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:49.526366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8970d0e-f03d-422b-9071-2b13b9db62c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:49.527135 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8970d0e-f03d-422b-9071-2b13b9db62c8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:49.527498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8970d0e-f03d-422b-9071-2b13b9db62c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:49.528098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8970d0e-f03d-422b-9071-2b13b9db62c8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:49.534807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cac2a-b7e5-4a98-bec4-2a0ebd133efb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:49.536507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cac2a-b7e5-4a98-bec4-2a0ebd133efb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:49.536771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8970d0e-f03d-422b-9071-2b13b9db62c8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:49.538775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8970d0e-f03d-422b-9071-2b13b9db62c8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-716cac2a-b7e5-4a98-bec4-2a0ebd133efb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:49.549518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:49.550543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:49.551465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:49.551465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:49.551465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:49.551465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:49.551465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:49.551465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:49.551465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:49.551465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:49.551465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 04:15:49.551465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d8970d0e-f03d-422b-9071-2b13b9db62c8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:49.552002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:49.552831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:49.554426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:49.554426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:49.554426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:49.554426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:49.554426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:49.554426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:49.554426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:49.554426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:49.554426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 04:15:49.554426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:49.556909 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 327/658] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:49.561602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 331/658] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:49.562441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cac2a-b7e5-4a98-bec4-2a0ebd133efb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:49.563423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8970d0e-f03d-422b-9071-2b13b9db62c8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:49.629906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:49.677277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cac2a-b7e5-4a98-bec4-2a0ebd133efb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:49.694351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8970d0e-f03d-422b-9071-2b13b9db62c8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:49.717470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:49.724768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b56e3-a917-4cf6-8885-e95f4fbbfd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:49.725677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b56e3-a917-4cf6-8885-e95f4fbbfd6f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:49.725955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b56e3-a917-4cf6-8885-e95f4fbbfd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:49.726220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b56e3-a917-4cf6-8885-e95f4fbbfd6f None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:49.726865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b56e3-a917-4cf6-8885-e95f4fbbfd6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:49.727163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b56e3-a917-4cf6-8885-e95f4fbbfd6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:49.729810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b56e3-a917-4cf6-8885-e95f4fbbfd6f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:49.733622 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8daf750-157e-48ff-8adf-464cc7204031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:49.734515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8daf750-157e-48ff-8adf-464cc7204031 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:49.735025 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8daf750-157e-48ff-8adf-464cc7204031 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:49.735760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8daf750-157e-48ff-8adf-464cc7204031 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:49.736708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8daf750-157e-48ff-8adf-464cc7204031 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:49.737243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8daf750-157e-48ff-8adf-464cc7204031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:49.738937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8daf750-157e-48ff-8adf-464cc7204031 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:49.744167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 328/659] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:49.745130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b56e3-a917-4cf6-8885-e95f4fbbfd6f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:49.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 332/660] 10.222.0.60 () {56 vars in 1139 bytes} [Sat Sep 17 04:15:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:49.766194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8daf750-157e-48ff-8adf-464cc7204031 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:49.872050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b56e3-a917-4cf6-8885-e95f4fbbfd6f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:49.882558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8daf750-157e-48ff-8adf-464cc7204031 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:49.898913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:49.911883 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c986f2-e282-4a45-a228-23c6e36c4cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:49.912846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c986f2-e282-4a45-a228-23c6e36c4cfd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:49.913439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c986f2-e282-4a45-a228-23c6e36c4cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:49.913885 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c986f2-e282-4a45-a228-23c6e36c4cfd None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:49.914790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c986f2-e282-4a45-a228-23c6e36c4cfd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:49.915270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c986f2-e282-4a45-a228-23c6e36c4cfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:49.917152 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c986f2-e282-4a45-a228-23c6e36c4cfd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:49.925257 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a62d3-aede-43db-81b6-34d3ddc7285c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:49.925941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a62d3-aede-43db-81b6-34d3ddc7285c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:49.926217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a62d3-aede-43db-81b6-34d3ddc7285c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:49.926485 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a62d3-aede-43db-81b6-34d3ddc7285c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:49.941760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a62d3-aede-43db-81b6-34d3ddc7285c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:49.941760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a62d3-aede-43db-81b6-34d3ddc7285c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:49.942116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 329/661] 10.222.0.60 () {56 vars in 1137 bytes} [Sat Sep 17 04:15:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:49.942858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c986f2-e282-4a45-a228-23c6e36c4cfd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-241a62d3-aede-43db-81b6-34d3ddc7285c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:49.955461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:49.956398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:49.957327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:49.957327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:49.957327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:49.957327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:49.957327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:49.957327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:49.957327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:49.957327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:49.959293 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 333/662] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:49.960307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a62d3-aede-43db-81b6-34d3ddc7285c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:50.003512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:50.073944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c986f2-e282-4a45-a228-23c6e36c4cfd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:50.074528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a62d3-aede-43db-81b6-34d3ddc7285c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:50.109964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6453-83e6-4c76-85b9-d6b65e37d730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:50.110569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6453-83e6-4c76-85b9-d6b65e37d730 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:50.110839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6453-83e6-4c76-85b9-d6b65e37d730 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:50.111103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6453-83e6-4c76-85b9-d6b65e37d730 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:50.125409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a7ee4-7576-4218-9c23-8daac0081d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:50.126460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a7ee4-7576-4218-9c23-8daac0081d84 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:50.126885 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a7ee4-7576-4218-9c23-8daac0081d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:50.127351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a7ee4-7576-4218-9c23-8daac0081d84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:50.161311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:50.183048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c6453-83e6-4c76-85b9-d6b65e37d730 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:50.184766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c6453-83e6-4c76-85b9-d6b65e37d730 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:50.203075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a7ee4-7576-4218-9c23-8daac0081d84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:50.204560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a7ee4-7576-4218-9c23-8daac0081d84 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:50.245927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:50.286414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 330/663] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:50.287538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6453-83e6-4c76-85b9-d6b65e37d730 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:50.317340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 334/664] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:50.318525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a7ee4-7576-4218-9c23-8daac0081d84 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:50.372985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:50.393415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6453-83e6-4c76-85b9-d6b65e37d730 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:50.429555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b107800-2d0a-4801-bc7b-ba9387815003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:50.430399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b107800-2d0a-4801-bc7b-ba9387815003 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:50.430839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b107800-2d0a-4801-bc7b-ba9387815003 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:50.431297 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b107800-2d0a-4801-bc7b-ba9387815003 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:50.432208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b107800-2d0a-4801-bc7b-ba9387815003 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:50.432643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b107800-2d0a-4801-bc7b-ba9387815003 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:50.434222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b107800-2d0a-4801-bc7b-ba9387815003 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:50.434444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a7ee4-7576-4218-9c23-8daac0081d84 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:50.447129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 331/665] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:50.448313 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b107800-2d0a-4801-bc7b-ba9387815003 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:50.473006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34b185-f100-4b68-9bf1-950c2307caeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:50.473598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34b185-f100-4b68-9bf1-950c2307caeb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:50.473876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34b185-f100-4b68-9bf1-950c2307caeb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:50.474142 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34b185-f100-4b68-9bf1-950c2307caeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:50.482689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:50.485920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34b185-f100-4b68-9bf1-950c2307caeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:50.487809 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34b185-f100-4b68-9bf1-950c2307caeb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-dc34b185-f100-4b68-9bf1-950c2307caeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:50.503936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:50.504981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:50.506070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:50.506070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:50.506070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:50.506070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 335/666] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:50.506070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34b185-f100-4b68-9bf1-950c2307caeb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:50.555905 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b107800-2d0a-4801-bc7b-ba9387815003 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:50.591029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4def4c5-c124-466e-b51e-307a1b6ca126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:50.591666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4def4c5-c124-466e-b51e-307a1b6ca126 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:50.591941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4def4c5-c124-466e-b51e-307a1b6ca126 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:50.592434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4def4c5-c124-466e-b51e-307a1b6ca126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:50.603947 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4def4c5-c124-466e-b51e-307a1b6ca126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:50.605518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4def4c5-c124-466e-b51e-307a1b6ca126 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:50.607520 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:50.612289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34b185-f100-4b68-9bf1-950c2307caeb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b4def4c5-c124-466e-b51e-307a1b6ca126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:50.622856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:50.624015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:50.625237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:50.625237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:50.625237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:50.628660 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 332/667] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:50.629923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d3178-c98c-4747-a6b4-a8c253ab925e None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:50.630208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d3178-c98c-4747-a6b4-a8c253ab925e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:50.630481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d3178-c98c-4747-a6b4-a8c253ab925e None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:50.631870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 333/668] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:50.632411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d3178-c98c-4747-a6b4-a8c253ab925e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:50.653196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a99d4-d14a-494f-9aec-cfc16d7b882e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:50.653969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a99d4-d14a-494f-9aec-cfc16d7b882e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:50.654242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a99d4-d14a-494f-9aec-cfc16d7b882e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:50.654635 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a99d4-d14a-494f-9aec-cfc16d7b882e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:50.655441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a99d4-d14a-494f-9aec-cfc16d7b882e None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=acf8a0ae03c94aa78cc2ab0e4e1c87b6) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:50.655751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a99d4-d14a-494f-9aec-cfc16d7b882e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:50.657316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a99d4-d14a-494f-9aec-cfc16d7b882e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:50.677114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 336/669] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:50 2022] GET /identity/v3/role_assignments?user.id=acf8a0ae03c94aa78cc2ab0e4e1c87b6&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:50.677114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a99d4-d14a-494f-9aec-cfc16d7b882e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:50.749855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:50.760051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d3178-c98c-4747-a6b4-a8c253ab925e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:50.797327 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624769ac-2aa9-498d-8052-c3c277993485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:50.798022 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624769ac-2aa9-498d-8052-c3c277993485 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:50.798357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624769ac-2aa9-498d-8052-c3c277993485 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:50.798632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624769ac-2aa9-498d-8052-c3c277993485 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:50.799380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624769ac-2aa9-498d-8052-c3c277993485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:50.799692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624769ac-2aa9-498d-8052-c3c277993485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:50.801417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624769ac-2aa9-498d-8052-c3c277993485 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:50.813062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a99d4-d14a-494f-9aec-cfc16d7b882e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:50.815244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 334/670] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:50.817214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0ed0c-b1fb-476d-8226-2a702684eb9e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:50.817214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0ed0c-b1fb-476d-8226-2a702684eb9e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:50.817214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0ed0c-b1fb-476d-8226-2a702684eb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:50.840675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-48a0ed0c-b1fb-476d-8226-2a702684eb9e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:50.843001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-48a0ed0c-b1fb-476d-8226-2a702684eb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:50.855122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3e23b-70f6-4978-90de-05975caa993a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:50.855452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3e23b-70f6-4978-90de-05975caa993a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:50.855713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3e23b-70f6-4978-90de-05975caa993a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:50.855999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3e23b-70f6-4978-90de-05975caa993a None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:50.856652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3e23b-70f6-4978-90de-05975caa993a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:50.856953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3e23b-70f6-4978-90de-05975caa993a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:50.858445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3e23b-70f6-4978-90de-05975caa993a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:50.871801 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 337/671] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:50.872642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3e23b-70f6-4978-90de-05975caa993a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:50.916594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:50.939974 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 335/672] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:50.942052 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0ed0c-b1fb-476d-8226-2a702684eb9e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:51.006244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3e23b-70f6-4978-90de-05975caa993a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:51.031336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:51.056940 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf09f51-4e2c-4a42-b69a-846630530921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:51.057851 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0ed0c-b1fb-476d-8226-2a702684eb9e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:51.058038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf09f51-4e2c-4a42-b69a-846630530921 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:51.058451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf09f51-4e2c-4a42-b69a-846630530921 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:51.058856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf09f51-4e2c-4a42-b69a-846630530921 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:51.066292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf09f51-4e2c-4a42-b69a-846630530921 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:51.068073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf09f51-4e2c-4a42-b69a-846630530921 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5cf09f51-4e2c-4a42-b69a-846630530921 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:51.081882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:51.082946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:51.084028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:51.084028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:51.084028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:51.084028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:51.084028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:51.084028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:51.084028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:51.084028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:51.086192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 338/673] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:51.094321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf09f51-4e2c-4a42-b69a-846630530921 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:51.097417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b636665-b3ad-4ca9-8789-a036a1c9fd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:51.098070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b636665-b3ad-4ca9-8789-a036a1c9fd75 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:51.098350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b636665-b3ad-4ca9-8789-a036a1c9fd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:51.098624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b636665-b3ad-4ca9-8789-a036a1c9fd75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:51.106591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b636665-b3ad-4ca9-8789-a036a1c9fd75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:51.108073 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b636665-b3ad-4ca9-8789-a036a1c9fd75 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-6b636665-b3ad-4ca9-8789-a036a1c9fd75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:51.125107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:51.126059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:51.126946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:51.126946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:51.126946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:51.126946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:51.126946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:51.126946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:51.126946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:51.126946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:51.126946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 336/674] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:51.127428 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b636665-b3ad-4ca9-8789-a036a1c9fd75 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:51.226868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf09f51-4e2c-4a42-b69a-846630530921 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:51.264921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:51.269965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a957e-6a59-4154-857f-3eddc02b124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:51.271484 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a957e-6a59-4154-857f-3eddc02b124d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:51.271827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a957e-6a59-4154-857f-3eddc02b124d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:51.272165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a957e-6a59-4154-857f-3eddc02b124d None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:51.273366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a957e-6a59-4154-857f-3eddc02b124d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:51.273736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a957e-6a59-4154-857f-3eddc02b124d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:51.275375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a957e-6a59-4154-857f-3eddc02b124d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:51.290705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 339/675] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:51.291777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a957e-6a59-4154-857f-3eddc02b124d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:51.333188 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b636665-b3ad-4ca9-8789-a036a1c9fd75 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:51.333425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:51.368351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c50eb3-fc14-43ec-b22a-5033ca7ceca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:51.369036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c50eb3-fc14-43ec-b22a-5033ca7ceca2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:51.369342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c50eb3-fc14-43ec-b22a-5033ca7ceca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:51.369605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c50eb3-fc14-43ec-b22a-5033ca7ceca2 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:51.370296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c50eb3-fc14-43ec-b22a-5033ca7ceca2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:51.370599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c50eb3-fc14-43ec-b22a-5033ca7ceca2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:51.372039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c50eb3-fc14-43ec-b22a-5033ca7ceca2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:51.384424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 337/676] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:51.385365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c50eb3-fc14-43ec-b22a-5033ca7ceca2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:51.407672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a957e-6a59-4154-857f-3eddc02b124d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:51.453170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7398d80-f1e5-48e7-b08c-93501ff7d7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:51.454478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398d80-f1e5-48e7-b08c-93501ff7d7eb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:51.454768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398d80-f1e5-48e7-b08c-93501ff7d7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:51.455043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398d80-f1e5-48e7-b08c-93501ff7d7eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:51.472065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7398d80-f1e5-48e7-b08c-93501ff7d7eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a7398d80-f1e5-48e7-b08c-93501ff7d7eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:51.478178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:51.479139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:51.479977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:51.479977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:51.479977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:51.479977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:51.479977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:51.479977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:51.479977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:51.479977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:51.479977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:51.479977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:51.479977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:51.479977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:51.482337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 340/677] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:51.483399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7398d80-f1e5-48e7-b08c-93501ff7d7eb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:51.495501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c50eb3-fc14-43ec-b22a-5033ca7ceca2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:51.505814 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:51.534742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fba76e-1ec0-4e28-99ad-86a520954ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:51.535346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fba76e-1ec0-4e28-99ad-86a520954ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:51.535620 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fba76e-1ec0-4e28-99ad-86a520954ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:51.535887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fba76e-1ec0-4e28-99ad-86a520954ed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:51.536606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fba76e-1ec0-4e28-99ad-86a520954ed9 None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=c9360be7ee9e4739b20c59955d28da47) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:51.536914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fba76e-1ec0-4e28-99ad-86a520954ed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:51.538400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fba76e-1ec0-4e28-99ad-86a520954ed9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:51.544655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 338/678] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:51 2022] GET /identity/v3/role_assignments?user.id=c9360be7ee9e4739b20c59955d28da47&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:51.546091 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fba76e-1ec0-4e28-99ad-86a520954ed9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:51.622045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:51.631820 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7398d80-f1e5-48e7-b08c-93501ff7d7eb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:51.690382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8352f4c-b4b9-48ce-b9f4-5fab777cbfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:51.691913 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8352f4c-b4b9-48ce-b9f4-5fab777cbfec None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:51.692397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8352f4c-b4b9-48ce-b9f4-5fab777cbfec None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:51.693197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8352f4c-b4b9-48ce-b9f4-5fab777cbfec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:51.694456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8352f4c-b4b9-48ce-b9f4-5fab777cbfec None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=6adf758d5fb9430189ee9057b4b7ea97) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:51.694971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8352f4c-b4b9-48ce-b9f4-5fab777cbfec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:51.697266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8352f4c-b4b9-48ce-b9f4-5fab777cbfec None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:51.705129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 341/679] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:51 2022] GET /identity/v3/role_assignments?user.id=6adf758d5fb9430189ee9057b4b7ea97&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:51.707593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45d722-73dd-4911-a990-7ddeccc1ba8c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:51.708088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45d722-73dd-4911-a990-7ddeccc1ba8c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:51.708558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45d722-73dd-4911-a990-7ddeccc1ba8c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:51.710775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 342/680] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:51.719907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc7403-d740-4e79-b601-92de655a67f6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:51.719907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc7403-d740-4e79-b601-92de655a67f6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:51.719907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc7403-d740-4e79-b601-92de655a67f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:51.775708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fba76e-1ec0-4e28-99ad-86a520954ed9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:51.795056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-00cc7403-d740-4e79-b601-92de655a67f6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:51.795056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-00cc7403-d740-4e79-b601-92de655a67f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:51.849168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e928add-5d0d-4699-9032-4e688c00dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:51.849168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e928add-5d0d-4699-9032-4e688c00dc9b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:51.849168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e928add-5d0d-4699-9032-4e688c00dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:51.849168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e928add-5d0d-4699-9032-4e688c00dc9b None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:51.849629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e928add-5d0d-4699-9032-4e688c00dc9b None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:51.849629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e928add-5d0d-4699-9032-4e688c00dc9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:51.852836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e928add-5d0d-4699-9032-4e688c00dc9b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:51.867492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 339/681] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:51.874776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e928add-5d0d-4699-9032-4e688c00dc9b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:51.948859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 343/682] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:51.962764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:52.096120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e928add-5d0d-4699-9032-4e688c00dc9b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:52.148458 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77873f-a8e3-45ec-ae68-fa09905c2910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:52.149121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77873f-a8e3-45ec-ae68-fa09905c2910 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:52.149413 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77873f-a8e3-45ec-ae68-fa09905c2910 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:52.149677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77873f-a8e3-45ec-ae68-fa09905c2910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:52.175248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77873f-a8e3-45ec-ae68-fa09905c2910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:52.176718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77873f-a8e3-45ec-ae68-fa09905c2910 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-ea77873f-a8e3-45ec-ae68-fa09905c2910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:52.200590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:52.201686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:52.203090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:52.203090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:52.203090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:52.203090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 340/683] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:52.208210 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc7403-d740-4e79-b601-92de655a67f6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:52.232748 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77873f-a8e3-45ec-ae68-fa09905c2910 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:52.340507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:52.358679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77873f-a8e3-45ec-ae68-fa09905c2910 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:52.393459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:52.402682 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde547b5-2f7e-45da-a080-17570f0961e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:52.402975 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc7403-d740-4e79-b601-92de655a67f6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:52.403338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde547b5-2f7e-45da-a080-17570f0961e8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:52.403635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde547b5-2f7e-45da-a080-17570f0961e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:52.403902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde547b5-2f7e-45da-a080-17570f0961e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:52.423324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde547b5-2f7e-45da-a080-17570f0961e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-dde547b5-2f7e-45da-a080-17570f0961e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:52.428567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:52.429533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:52.430734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 341/684] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:15:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:52.435806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde547b5-2f7e-45da-a080-17570f0961e8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:52.461140 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82329fa6-5f30-405b-9f5f-415b8e7f76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:52.462209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82329fa6-5f30-405b-9f5f-415b8e7f76c9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:52.462807 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82329fa6-5f30-405b-9f5f-415b8e7f76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:52.463303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82329fa6-5f30-405b-9f5f-415b8e7f76c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:52.464303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82329fa6-5f30-405b-9f5f-415b8e7f76c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:52.464797 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82329fa6-5f30-405b-9f5f-415b8e7f76c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:52.466535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82329fa6-5f30-405b-9f5f-415b8e7f76c9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:52.490222 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 344/685] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:52.496228 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82329fa6-5f30-405b-9f5f-415b8e7f76c9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:52.551767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde547b5-2f7e-45da-a080-17570f0961e8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:52.622356 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a5502-1141-4e02-8766-eb563a3e96f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:52.623287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a5502-1141-4e02-8766-eb563a3e96f0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:52.623739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a5502-1141-4e02-8766-eb563a3e96f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:52.624195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a5502-1141-4e02-8766-eb563a3e96f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:52.625033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a5502-1141-4e02-8766-eb563a3e96f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:52.625529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a5502-1141-4e02-8766-eb563a3e96f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:52.627062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a5502-1141-4e02-8766-eb563a3e96f0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:52.652190 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 342/686] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:52.653410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a5502-1141-4e02-8766-eb563a3e96f0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:52.670210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:52.690392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82329fa6-5f30-405b-9f5f-415b8e7f76c9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:52.726031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22a282-2ca7-4331-9b95-d9d7e8d3cdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:52.726659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22a282-2ca7-4331-9b95-d9d7e8d3cdce None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:52.726944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22a282-2ca7-4331-9b95-d9d7e8d3cdce None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:52.727213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22a282-2ca7-4331-9b95-d9d7e8d3cdce None admin] PATH_INFO: `/v3/users/swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:52.735418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22a282-2ca7-4331-9b95-d9d7e8d3cdce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:52.736838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22a282-2ca7-4331-9b95-d9d7e8d3cdce None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ea22a282-2ca7-4331-9b95-d9d7e8d3cdce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:52.748958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:52.750017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:52.751409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:52.751409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:52.751409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:52.751409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:52.751409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:52.751409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:52.751409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:52.751409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:52.751409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 04:15:52.751409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:52.751409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 345/687] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:15:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:52.756307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22a282-2ca7-4331-9b95-d9d7e8d3cdce None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:52.770528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:52.795764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a5502-1141-4e02-8766-eb563a3e96f0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:52.864289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22a282-2ca7-4331-9b95-d9d7e8d3cdce None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:52.900868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aae00f-9565-4039-870f-86fbe091126b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:52.901709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aae00f-9565-4039-870f-86fbe091126b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:52.902087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aae00f-9565-4039-870f-86fbe091126b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:52.902454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aae00f-9565-4039-870f-86fbe091126b None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:52.903291 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aae00f-9565-4039-870f-86fbe091126b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:52.903698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aae00f-9565-4039-870f-86fbe091126b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:52.905267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aae00f-9565-4039-870f-86fbe091126b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:52.921809 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b2e42-4e7b-483e-a4b3-d4541377afcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:52.922466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b2e42-4e7b-483e-a4b3-d4541377afcf None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:52.922746 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b2e42-4e7b-483e-a4b3-d4541377afcf None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:52.923018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b2e42-4e7b-483e-a4b3-d4541377afcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:52.923610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 346/688] 10.222.0.60 () {56 vars in 1135 bytes} [Sat Sep 17 04:15:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:52.929483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aae00f-9565-4039-870f-86fbe091126b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:52.954090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b2e42-4e7b-483e-a4b3-d4541377afcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:52.955575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b2e42-4e7b-483e-a4b3-d4541377afcf None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c68b2e42-4e7b-483e-a4b3-d4541377afcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:52.998279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:53.000319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:53.002543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:53.002543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:53.002543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:53.003500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:53.003915 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 343/689] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:53.014100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b2e42-4e7b-483e-a4b3-d4541377afcf None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:53.047621 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aae00f-9565-4039-870f-86fbe091126b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:53.107417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d685d-f1d0-45ac-ab5e-871102fedaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:53.108041 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d685d-f1d0-45ac-ab5e-871102fedaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:53.108314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d685d-f1d0-45ac-ab5e-871102fedaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:53.108583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d685d-f1d0-45ac-ab5e-871102fedaa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:53.136095 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:53.222832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d685d-f1d0-45ac-ab5e-871102fedaa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:53.224200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d685d-f1d0-45ac-ab5e-871102fedaa1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:53.279020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b2e42-4e7b-483e-a4b3-d4541377afcf None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:53.301541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:53.325304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 347/690] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:53.327409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d685d-f1d0-45ac-ab5e-871102fedaa1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:53.404052 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ef63c-c2b6-430f-abab-cae76c3184cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:53.404672 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ef63c-c2b6-430f-abab-cae76c3184cd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:53.404948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ef63c-c2b6-430f-abab-cae76c3184cd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:53.405242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ef63c-c2b6-430f-abab-cae76c3184cd None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:53.405891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ef63c-c2b6-430f-abab-cae76c3184cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:53.406187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ef63c-c2b6-430f-abab-cae76c3184cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:53.407559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ef63c-c2b6-430f-abab-cae76c3184cd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:53.429037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d685d-f1d0-45ac-ab5e-871102fedaa1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:53.442410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 344/691] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:53.447923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ef63c-c2b6-430f-abab-cae76c3184cd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:53.462819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c425a9b-df8f-42ae-8b56-f4b176bba714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:53.463455 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c425a9b-df8f-42ae-8b56-f4b176bba714 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:53.463734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c425a9b-df8f-42ae-8b56-f4b176bba714 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:53.464005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c425a9b-df8f-42ae-8b56-f4b176bba714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:53.474474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c425a9b-df8f-42ae-8b56-f4b176bba714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:53.475869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c425a9b-df8f-42ae-8b56-f4b176bba714 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6c425a9b-df8f-42ae-8b56-f4b176bba714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:53.490944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:53.492512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:53.493548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:53.493548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:53.493548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:53.493548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 348/692] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:53.494066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b471b-fce2-4c49-9f86-c80e24025fdf None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:53.494341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b471b-fce2-4c49-9f86-c80e24025fdf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:53.494605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b471b-fce2-4c49-9f86-c80e24025fdf None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:53.496046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 349/693] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:53.502643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b471b-fce2-4c49-9f86-c80e24025fdf None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:53.528327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:53.586619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ef63c-c2b6-430f-abab-cae76c3184cd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:53.605312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:53.615450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b471b-fce2-4c49-9f86-c80e24025fdf None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:53.622600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30728a2b-f0b6-4439-91d0-34d5ef8ec197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:53.623343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728a2b-f0b6-4439-91d0-34d5ef8ec197 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:53.623732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728a2b-f0b6-4439-91d0-34d5ef8ec197 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:53.624093 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728a2b-f0b6-4439-91d0-34d5ef8ec197 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:53.631543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30728a2b-f0b6-4439-91d0-34d5ef8ec197 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:53.633272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30728a2b-f0b6-4439-91d0-34d5ef8ec197 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-30728a2b-f0b6-4439-91d0-34d5ef8ec197 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:53.645466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:53.646643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:53.647811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:53.647811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:53.647811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:53.647811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:53.647811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:53.647811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:53.647811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:53.647811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:53.647811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:15:53.647811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:53.649448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 345/694] 10.222.0.60 () {56 vars in 1093 bytes} [Sat Sep 17 04:15:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:53.651053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6164b-f6da-46d5-a982-7791d1819170 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:53.651421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6164b-f6da-46d5-a982-7791d1819170 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:53.651776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6164b-f6da-46d5-a982-7791d1819170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:53.660507 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629bfb1-97c1-4824-8256-788f7e1d2ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:53.661357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629bfb1-97c1-4824-8256-788f7e1d2ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:53.661804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629bfb1-97c1-4824-8256-788f7e1d2ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:53.662216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629bfb1-97c1-4824-8256-788f7e1d2ad4 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:53.663002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629bfb1-97c1-4824-8256-788f7e1d2ad4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:53.663440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629bfb1-97c1-4824-8256-788f7e1d2ad4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:53.665153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629bfb1-97c1-4824-8256-788f7e1d2ad4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:53.671259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2ed6164b-f6da-46d5-a982-7791d1819170 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:53.672541 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2ed6164b-f6da-46d5-a982-7791d1819170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:53.681411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 350/695] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:53.682547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629bfb1-97c1-4824-8256-788f7e1d2ad4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:53.760671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 346/696] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:53.761741 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6164b-f6da-46d5-a982-7791d1819170 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:53.769928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:53.791262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629bfb1-97c1-4824-8256-788f7e1d2ad4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:53.829883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66b0cd-ddf2-47f5-92b2-58dfda88d82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:53.830861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66b0cd-ddf2-47f5-92b2-58dfda88d82d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:53.831365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66b0cd-ddf2-47f5-92b2-58dfda88d82d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:53.831828 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66b0cd-ddf2-47f5-92b2-58dfda88d82d None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:53.832772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66b0cd-ddf2-47f5-92b2-58dfda88d82d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:53.833422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66b0cd-ddf2-47f5-92b2-58dfda88d82d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:53.835429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66b0cd-ddf2-47f5-92b2-58dfda88d82d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:53.859740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 351/697] 10.222.0.60 () {56 vars in 1133 bytes} [Sat Sep 17 04:15:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:53.865422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66b0cd-ddf2-47f5-92b2-58dfda88d82d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:53.941897 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6164b-f6da-46d5-a982-7791d1819170 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:53.951115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:53.991808 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da3d51-d2b4-4ed0-8212-c9cacc1256cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:53.992486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da3d51-d2b4-4ed0-8212-c9cacc1256cf None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:53.992760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da3d51-d2b4-4ed0-8212-c9cacc1256cf None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:53.993024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da3d51-d2b4-4ed0-8212-c9cacc1256cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.014024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da3d51-d2b4-4ed0-8212-c9cacc1256cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:54.015417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da3d51-d2b4-4ed0-8212-c9cacc1256cf None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-94da3d51-d2b4-4ed0-8212-c9cacc1256cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:54.039564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:54.040631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:54.042505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:54.042505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:54.042505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:54.042505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:54.042505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:54.042505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:54.042505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:54.042505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:54.042505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 347/698] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:54.042505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da3d51-d2b4-4ed0-8212-c9cacc1256cf None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:54.054796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66b0cd-ddf2-47f5-92b2-58dfda88d82d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:54.098918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f813d80-0396-4fe5-9e75-5f84163bb132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:54.099539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f813d80-0396-4fe5-9e75-5f84163bb132 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:54.099814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f813d80-0396-4fe5-9e75-5f84163bb132 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:54.100084 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f813d80-0396-4fe5-9e75-5f84163bb132 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.111060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:54.142145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da3d51-d2b4-4ed0-8212-c9cacc1256cf None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:54.175070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f813d80-0396-4fe5-9e75-5f84163bb132 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:54.176472 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc7435-6335-436c-858d-e235949b1784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:54.176631 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f813d80-0396-4fe5-9e75-5f84163bb132 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:54.177126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc7435-6335-436c-858d-e235949b1784 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:54.177402 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc7435-6335-436c-858d-e235949b1784 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:54.177670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc7435-6335-436c-858d-e235949b1784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.188604 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc7435-6335-436c-858d-e235949b1784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:54.190070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc7435-6335-436c-858d-e235949b1784 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-4ccc7435-6335-436c-858d-e235949b1784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:54.205195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:54.206154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:54.207033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:54.207033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:54.207033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:54.207033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 348/699] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:54.211895 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc7435-6335-436c-858d-e235949b1784 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:54.231418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:54.285123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 352/700] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 04:15:54.285123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3abfcd-e0bb-4d80-8611-7dbf016c2a6e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:54.285123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3abfcd-e0bb-4d80-8611-7dbf016c2a6e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:54.285123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3abfcd-e0bb-4d80-8611-7dbf016c2a6e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.288345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 353/701] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:54.288983 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3abfcd-e0bb-4d80-8611-7dbf016c2a6e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:54.321957 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc7435-6335-436c-858d-e235949b1784 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:54.334892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:54.357894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17489d7-9aa7-46a6-a8ce-bb69fb2c0784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:54.358527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17489d7-9aa7-46a6-a8ce-bb69fb2c0784 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:54.358806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17489d7-9aa7-46a6-a8ce-bb69fb2c0784 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:54.359074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17489d7-9aa7-46a6-a8ce-bb69fb2c0784 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.359777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17489d7-9aa7-46a6-a8ce-bb69fb2c0784 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:54.360084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17489d7-9aa7-46a6-a8ce-bb69fb2c0784 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:54.361588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17489d7-9aa7-46a6-a8ce-bb69fb2c0784 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:54.381119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 349/702] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:54.381119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17489d7-9aa7-46a6-a8ce-bb69fb2c0784 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:54.408846 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3abfcd-e0bb-4d80-8611-7dbf016c2a6e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:54.445979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecb78f-55ce-4cdd-b256-14d2d5bb9597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:54.447047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecb78f-55ce-4cdd-b256-14d2d5bb9597 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:54.447525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecb78f-55ce-4cdd-b256-14d2d5bb9597 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:54.447993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecb78f-55ce-4cdd-b256-14d2d5bb9597 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.448842 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecb78f-55ce-4cdd-b256-14d2d5bb9597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:54.449237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecb78f-55ce-4cdd-b256-14d2d5bb9597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:54.450613 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecb78f-55ce-4cdd-b256-14d2d5bb9597 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:54.457005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:54.463529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 354/703] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:54.464775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adc5b5-2b52-4f0c-aeb8-7406273b9e37 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:54.465060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adc5b5-2b52-4f0c-aeb8-7406273b9e37 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:54.465365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adc5b5-2b52-4f0c-aeb8-7406273b9e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.477865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17489d7-9aa7-46a6-a8ce-bb69fb2c0784 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:54.482901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e7adc5b5-2b52-4f0c-aeb8-7406273b9e37 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:54.484088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e7adc5b5-2b52-4f0c-aeb8-7406273b9e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:54.510746 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d9e33-c478-46ac-b9ba-4465af4fbc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:54.511384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d9e33-c478-46ac-b9ba-4465af4fbc48 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:54.511659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d9e33-c478-46ac-b9ba-4465af4fbc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:54.511929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d9e33-c478-46ac-b9ba-4465af4fbc48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.522603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d9e33-c478-46ac-b9ba-4465af4fbc48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:54.524042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d9e33-c478-46ac-b9ba-4465af4fbc48 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d78d9e33-c478-46ac-b9ba-4465af4fbc48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:54.539326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:54.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:54.546363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:54.546363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:54.546363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:54.546363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 350/704] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:54.546363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d9e33-c478-46ac-b9ba-4465af4fbc48 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:54.551186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 355/705] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:54.552473 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb07c-19b2-4b36-a8ab-bee6d4879c94 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:54.552814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb07c-19b2-4b36-a8ab-bee6d4879c94 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:54.553170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb07c-19b2-4b36-a8ab-bee6d4879c94 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.554696 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 356/706] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:54.555421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cb07c-19b2-4b36-a8ab-bee6d4879c94 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:54.583316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:54.647986 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d9e33-c478-46ac-b9ba-4465af4fbc48 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:54.670812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cb07c-19b2-4b36-a8ab-bee6d4879c94 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:54.679630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:54.686178 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870ff36-37cf-43a6-b65f-ce663a243ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:54.686784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870ff36-37cf-43a6-b65f-ce663a243ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:54.687254 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870ff36-37cf-43a6-b65f-ce663a243ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:54.687362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870ff36-37cf-43a6-b65f-ce663a243ae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.688045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870ff36-37cf-43a6-b65f-ce663a243ae2 None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=7aac7cd1742449328facf2902483f259) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:54.688398 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870ff36-37cf-43a6-b65f-ce663a243ae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:54.689789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870ff36-37cf-43a6-b65f-ce663a243ae2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:54.705131 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 351/707] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:54 2022] GET /identity/v3/role_assignments?user.id=7aac7cd1742449328facf2902483f259&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:54.705131 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870ff36-37cf-43a6-b65f-ce663a243ae2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:54.718923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87393b8-5b99-4b26-b0db-3a4a8bca7c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:54.719707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87393b8-5b99-4b26-b0db-3a4a8bca7c69 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:54.720053 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87393b8-5b99-4b26-b0db-3a4a8bca7c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:54.720432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87393b8-5b99-4b26-b0db-3a4a8bca7c69 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.721251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87393b8-5b99-4b26-b0db-3a4a8bca7c69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:54.723023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87393b8-5b99-4b26-b0db-3a4a8bca7c69 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:54.749071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 357/708] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:15:54 2022] POST /identity/v3/users => generated 254 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:15:54.750559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c925b9-6ada-4024-bfd9-0e5187c1fc86 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:54.750816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c925b9-6ada-4024-bfd9-0e5187c1fc86 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:54.751199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c925b9-6ada-4024-bfd9-0e5187c1fc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.772808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-40c925b9-6ada-4024-bfd9-0e5187c1fc86 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:54.774458 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-40c925b9-6ada-4024-bfd9-0e5187c1fc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:54.850737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870ff36-37cf-43a6-b65f-ce663a243ae2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:54.862257 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 358/709] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:54.863348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c925b9-6ada-4024-bfd9-0e5187c1fc86 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:54.888016 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:54.913975 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c42b35-309c-40f8-8f30-741456b2de1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:54.914630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c42b35-309c-40f8-8f30-741456b2de1c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:54.914907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c42b35-309c-40f8-8f30-741456b2de1c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:54.915193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c42b35-309c-40f8-8f30-741456b2de1c None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:54.915965 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c42b35-309c-40f8-8f30-741456b2de1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:54.916205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c42b35-309c-40f8-8f30-741456b2de1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:54.917677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c42b35-309c-40f8-8f30-741456b2de1c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:54.945197 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 352/710] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:54.949655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c42b35-309c-40f8-8f30-741456b2de1c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:54.988926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c925b9-6ada-4024-bfd9-0e5187c1fc86 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:55.023758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:55.036967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e6e67-53bc-4859-a34d-b35f61620733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:55.038008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e6e67-53bc-4859-a34d-b35f61620733 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:55.038493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e6e67-53bc-4859-a34d-b35f61620733 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:55.038964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e6e67-53bc-4859-a34d-b35f61620733 None admin] PATH_INFO: `/v3/services/network` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:55.039970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e6e67-53bc-4859-a34d-b35f61620733 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:55.042370 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e6e67-53bc-4859-a34d-b35f61620733 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d64e6e67-53bc-4859-a34d-b35f61620733 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:55.056276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:55.060579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:55.061523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 359/711] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:54 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:55.065691 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e6e67-53bc-4859-a34d-b35f61620733 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:55.089067 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c42b35-309c-40f8-8f30-741456b2de1c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:55.131120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ec353-2e1b-4ec4-913f-56ce3515d2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:55.131678 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ec353-2e1b-4ec4-913f-56ce3515d2af None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:55.132271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ec353-2e1b-4ec4-913f-56ce3515d2af None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:55.132543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ec353-2e1b-4ec4-913f-56ce3515d2af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:55.141379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ec353-2e1b-4ec4-913f-56ce3515d2af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:55.142990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ec353-2e1b-4ec4-913f-56ce3515d2af None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:55.155994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-6e9ec353-2e1b-4ec4-913f-56ce3515d2af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:15:55.157503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:55.158467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:55.159407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:15:55.159407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:15:55.159407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:15:55.159407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:15:55.159407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:15:55.159407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:15:55.159407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:15:55.159407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:55.159407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 353/712] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:55.164824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ec353-2e1b-4ec4-913f-56ce3515d2af None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:55.184059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e6e67-53bc-4859-a34d-b35f61620733 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:55.224249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091d6ed-14da-4613-ade1-498dcad24ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:55.225174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091d6ed-14da-4613-ade1-498dcad24ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:55.225582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091d6ed-14da-4613-ade1-498dcad24ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:55.225863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091d6ed-14da-4613-ade1-498dcad24ed8 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:55.226501 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091d6ed-14da-4613-ade1-498dcad24ed8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:55.226803 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091d6ed-14da-4613-ade1-498dcad24ed8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:55.228533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091d6ed-14da-4613-ade1-498dcad24ed8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:55.235376 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 360/713] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:15:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:55.242649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091d6ed-14da-4613-ade1-498dcad24ed8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:55.286654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ec353-2e1b-4ec4-913f-56ce3515d2af None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:55.300947 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:55.323809 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acb3e5-88b5-4c4f-a6e1-4c66cf2a39ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:55.325170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acb3e5-88b5-4c4f-a6e1-4c66cf2a39ef None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:55.325605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acb3e5-88b5-4c4f-a6e1-4c66cf2a39ef None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:55.326009 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acb3e5-88b5-4c4f-a6e1-4c66cf2a39ef None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:55.327000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acb3e5-88b5-4c4f-a6e1-4c66cf2a39ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:55.327470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acb3e5-88b5-4c4f-a6e1-4c66cf2a39ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:55.329045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acb3e5-88b5-4c4f-a6e1-4c66cf2a39ef None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:55.342480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 354/714] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:55.343582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acb3e5-88b5-4c4f-a6e1-4c66cf2a39ef None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:55.355974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091d6ed-14da-4613-ade1-498dcad24ed8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:55.393019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdff3a-2465-4c91-a5ff-613894f26311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:55.393749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdff3a-2465-4c91-a5ff-613894f26311 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:55.394269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdff3a-2465-4c91-a5ff-613894f26311 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:55.394661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdff3a-2465-4c91-a5ff-613894f26311 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:55.395679 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdff3a-2465-4c91-a5ff-613894f26311 None admin] RBAC: Adding query filter params (type=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:55.396080 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdff3a-2465-4c91-a5ff-613894f26311 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:55.397640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdff3a-2465-4c91-a5ff-613894f26311 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:55.406104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 361/715] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:15:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:55.407107 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdff3a-2465-4c91-a5ff-613894f26311 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:55.418838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:55.454962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acb3e5-88b5-4c4f-a6e1-4c66cf2a39ef None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:55.492439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07917468-dd86-40a5-956f-aea1eedb0eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:55.493155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07917468-dd86-40a5-956f-aea1eedb0eeb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:55.493435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07917468-dd86-40a5-956f-aea1eedb0eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:55.493752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07917468-dd86-40a5-956f-aea1eedb0eeb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:55.494482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07917468-dd86-40a5-956f-aea1eedb0eeb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:55.495965 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07917468-dd86-40a5-956f-aea1eedb0eeb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-07917468-dd86-40a5-956f-aea1eedb0eeb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:55.508380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:55.509820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:55.510840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 355/716] 10.222.0.60 () {56 vars in 1129 bytes} [Sat Sep 17 04:15:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:55.515585 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07917468-dd86-40a5-956f-aea1eedb0eeb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:55.561048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdff3a-2465-4c91-a5ff-613894f26311 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:55.569446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:55.636551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07917468-dd86-40a5-956f-aea1eedb0eeb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:55.638626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf7747-6f54-46b9-83a8-03fb8d2ad430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:55.639521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf7747-6f54-46b9-83a8-03fb8d2ad430 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:55.639938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf7747-6f54-46b9-83a8-03fb8d2ad430 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:55.640632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf7747-6f54-46b9-83a8-03fb8d2ad430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:55.641578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf7747-6f54-46b9-83a8-03fb8d2ad430 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=d3272d57d5d24e5a990a6d7c86e6d4bc) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:55.642055 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf7747-6f54-46b9-83a8-03fb8d2ad430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:55.643698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf7747-6f54-46b9-83a8-03fb8d2ad430 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:55.659848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 362/717] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:15:55 2022] GET /identity/v3/role_assignments?user.id=d3272d57d5d24e5a990a6d7c86e6d4bc&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:55.683718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bda921-afa5-4038-8e88-e6800091561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:55.684362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bda921-afa5-4038-8e88-e6800091561b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:55.684637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bda921-afa5-4038-8e88-e6800091561b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:55.684907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bda921-afa5-4038-8e88-e6800091561b None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:55.685626 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bda921-afa5-4038-8e88-e6800091561b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:55.685930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bda921-afa5-4038-8e88-e6800091561b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:55.687356 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bda921-afa5-4038-8e88-e6800091561b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:55.693200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 356/718] 10.222.0.60 () {56 vars in 1124 bytes} [Sat Sep 17 04:15:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:55.698646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf7747-6f54-46b9-83a8-03fb8d2ad430 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:55.783440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:55.820941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf7747-6f54-46b9-83a8-03fb8d2ad430 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:55.872274 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60093d92-63bc-414b-846b-da56a87bfccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:55.872908 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60093d92-63bc-414b-846b-da56a87bfccf None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:55.873215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60093d92-63bc-414b-846b-da56a87bfccf None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:55.873494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60093d92-63bc-414b-846b-da56a87bfccf None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:55.874143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60093d92-63bc-414b-846b-da56a87bfccf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:55.874446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60093d92-63bc-414b-846b-da56a87bfccf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:55.875839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60093d92-63bc-414b-846b-da56a87bfccf None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:55.881327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 363/719] 10.222.0.60 () {56 vars in 1124 bytes} [Sat Sep 17 04:15:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:57.444870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f899c1-7d9e-4919-9b0e-2a4207635bdb None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:57.446048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f899c1-7d9e-4919-9b0e-2a4207635bdb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:57.446531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f899c1-7d9e-4919-9b0e-2a4207635bdb None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:57.448361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 357/720] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:57.454336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeef14-8e69-4f14-b161-01781404ab73 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:57.455267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeef14-8e69-4f14-b161-01781404ab73 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:57.455749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeef14-8e69-4f14-b161-01781404ab73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:57.475537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-caaeef14-8e69-4f14-b161-01781404ab73 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:57.477108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-caaeef14-8e69-4f14-b161-01781404ab73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:57.574763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 364/721] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:57.655876 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fa847-969d-42dc-8b55-ab63d0ddbaad None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:57.656207 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fa847-969d-42dc-8b55-ab63d0ddbaad None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:57.656425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fa847-969d-42dc-8b55-ab63d0ddbaad None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:57.661060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 358/722] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:57.666821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f51351-088d-48c5-8afc-b0b3b75934c2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:57.667106 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f51351-088d-48c5-8afc-b0b3b75934c2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:57.667374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f51351-088d-48c5-8afc-b0b3b75934c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:57.690965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a4f51351-088d-48c5-8afc-b0b3b75934c2 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:57.692152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a4f51351-088d-48c5-8afc-b0b3b75934c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:57.774833 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:57.783715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 365/723] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:57.963599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9ce3a-4b61-4755-8a43-0494b5a0885f None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:57.963977 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9ce3a-4b61-4755-8a43-0494b5a0885f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:57.964174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9ce3a-4b61-4755-8a43-0494b5a0885f None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:57.965797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 359/724] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:57.970708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f80518-bac0-496a-b152-cd2ddba7cd0c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:57.970992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f80518-bac0-496a-b152-cd2ddba7cd0c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:57.971264 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f80518-bac0-496a-b152-cd2ddba7cd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:57.981916 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9ce3a-4b61-4755-8a43-0494b5a0885f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:58.001375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-98f80518-bac0-496a-b152-cd2ddba7cd0c None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:58.002559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-98f80518-bac0-496a-b152-cd2ddba7cd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:58.087028 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9ce3a-4b61-4755-8a43-0494b5a0885f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:58.089102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:58.136659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 366/725] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 04:15:58.159985 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f80518-bac0-496a-b152-cd2ddba7cd0c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:58.166103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f42dee-9c23-4f29-8727-1a03d0fb0d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:58.167205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f42dee-9c23-4f29-8727-1a03d0fb0d0c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:58.167686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f42dee-9c23-4f29-8727-1a03d0fb0d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:58.168164 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f42dee-9c23-4f29-8727-1a03d0fb0d0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:58.196568 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f42dee-9c23-4f29-8727-1a03d0fb0d0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-13f42dee-9c23-4f29-8727-1a03d0fb0d0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:58.217702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:58.218670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:58.219505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 360/726] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:15:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:15:58.226497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f42dee-9c23-4f29-8727-1a03d0fb0d0c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:58.272714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f80518-bac0-496a-b152-cd2ddba7cd0c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:58.296588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:58.307387 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6169a8-befd-42a6-bdad-5c0017d426fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:58.308029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6169a8-befd-42a6-bdad-5c0017d426fa None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:58.308307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6169a8-befd-42a6-bdad-5c0017d426fa None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:58.308578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6169a8-befd-42a6-bdad-5c0017d426fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:58.338796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6169a8-befd-42a6-bdad-5c0017d426fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-2a6169a8-befd-42a6-bdad-5c0017d426fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:58.349954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:58.350968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:58.351836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 367/727] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:15:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:58.358215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6169a8-befd-42a6-bdad-5c0017d426fa None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:58.383590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f42dee-9c23-4f29-8727-1a03d0fb0d0c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:58.451013 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e6ef7-c279-4bab-af88-d56e6b331139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:58.451683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e6ef7-c279-4bab-af88-d56e6b331139 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:58.451916 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e6ef7-c279-4bab-af88-d56e6b331139 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:58.452148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e6ef7-c279-4bab-af88-d56e6b331139 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:58.452811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e6ef7-c279-4bab-af88-d56e6b331139 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:58.453115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e6ef7-c279-4bab-af88-d56e6b331139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:58.454487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e6ef7-c279-4bab-af88-d56e6b331139 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:58.468902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 361/728] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:15:58.470187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99da0d1-38c5-43b1-b112-a6da579c289a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:58.470469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99da0d1-38c5-43b1-b112-a6da579c289a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:58.470744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99da0d1-38c5-43b1-b112-a6da579c289a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:58.472162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 362/729] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:58.472847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99da0d1-38c5-43b1-b112-a6da579c289a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:58.500204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:58.513963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6169a8-befd-42a6-bdad-5c0017d426fa None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:58.550311 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93836536-8f75-4665-ba56-cf26ded9afd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:58.550935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93836536-8f75-4665-ba56-cf26ded9afd5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:58.551216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93836536-8f75-4665-ba56-cf26ded9afd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:58.551487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93836536-8f75-4665-ba56-cf26ded9afd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:58.552163 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93836536-8f75-4665-ba56-cf26ded9afd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:58.552461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93836536-8f75-4665-ba56-cf26ded9afd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:58.553869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93836536-8f75-4665-ba56-cf26ded9afd5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:58.562216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 368/730] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:15:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:58.563017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93836536-8f75-4665-ba56-cf26ded9afd5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:58.594871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99da0d1-38c5-43b1-b112-a6da579c289a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:58.609506 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:58.630454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a2f72-daf8-495c-9fbd-fc034748184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:58.631057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a2f72-daf8-495c-9fbd-fc034748184f None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:58.631334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a2f72-daf8-495c-9fbd-fc034748184f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:58.631601 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a2f72-daf8-495c-9fbd-fc034748184f None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:58.632250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a2f72-daf8-495c-9fbd-fc034748184f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:58.633640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a2f72-daf8-495c-9fbd-fc034748184f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:58.657762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 363/731] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:15:58 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:15:58.657762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c80ada-b8d2-45bb-be45-324e494a871a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:58.657762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c80ada-b8d2-45bb-be45-324e494a871a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:58.657762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c80ada-b8d2-45bb-be45-324e494a871a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:58.674618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-69c80ada-b8d2-45bb-be45-324e494a871a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:58.675924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-69c80ada-b8d2-45bb-be45-324e494a871a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:58.676977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93836536-8f75-4665-ba56-cf26ded9afd5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:58.721505 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74678f-083e-470a-9f5c-a909714efc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:58.722157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74678f-083e-470a-9f5c-a909714efc8c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:58.722431 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74678f-083e-470a-9f5c-a909714efc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:58.722702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74678f-083e-470a-9f5c-a909714efc8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:58.734390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74678f-083e-470a-9f5c-a909714efc8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:58.735896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74678f-083e-470a-9f5c-a909714efc8c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:58.753510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 364/732] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:58 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:58.753510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c80ada-b8d2-45bb-be45-324e494a871a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-df74678f-083e-470a-9f5c-a909714efc8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:58.753799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:58.754798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:58.755676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:58.755676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:58.755676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:58.755676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 369/733] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:58.759154 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2920af-a08c-41d0-b0ff-055fb511c95d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:58.759443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2920af-a08c-41d0-b0ff-055fb511c95d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:58.759718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2920af-a08c-41d0-b0ff-055fb511c95d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:58.765314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 370/734] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:15:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:15:58.765314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2920af-a08c-41d0-b0ff-055fb511c95d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:58.778642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:58.884320 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c80ada-b8d2-45bb-be45-324e494a871a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:58.888049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:58.896933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2920af-a08c-41d0-b0ff-055fb511c95d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:58.921388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e06c7-553b-42c6-b1d4-c76018a3f082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:58.922018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e06c7-553b-42c6-b1d4-c76018a3f082 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:58.922290 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e06c7-553b-42c6-b1d4-c76018a3f082 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:58.922555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e06c7-553b-42c6-b1d4-c76018a3f082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:58.932410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e06c7-553b-42c6-b1d4-c76018a3f082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:58.933957 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e06c7-553b-42c6-b1d4-c76018a3f082 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:58.938676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a13c8-8c12-486c-b4e7-b9d1ab44bc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:58.939527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a13c8-8c12-486c-b4e7-b9d1ab44bc8f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:58.939998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a13c8-8c12-486c-b4e7-b9d1ab44bc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:58.940460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a13c8-8c12-486c-b4e7-b9d1ab44bc8f None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:58.942221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a13c8-8c12-486c-b4e7-b9d1ab44bc8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:58.943284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a13c8-8c12-486c-b4e7-b9d1ab44bc8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:58.945723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a13c8-8c12-486c-b4e7-b9d1ab44bc8f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-1a8e06c7-553b-42c6-b1d4-c76018a3f082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:15:58.948991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:58.949929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:15:58.951127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:15:58.951127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:15:58.951127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:58.951127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 365/735] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:58.952459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b403dc9-8635-4a01-aad1-f24b16036bdb None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:58.952836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b403dc9-8635-4a01-aad1-f24b16036bdb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:58.953242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b403dc9-8635-4a01-aad1-f24b16036bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:58.964324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 371/736] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:58.965515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a13c8-8c12-486c-b4e7-b9d1ab44bc8f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:58.971904 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2b403dc9-8635-4a01-aad1-f24b16036bdb None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:15:58.973303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2b403dc9-8635-4a01-aad1-f24b16036bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:15:59.048155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:59.057425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 366/737] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:15:58 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 04:15:59.058606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b403dc9-8635-4a01-aad1-f24b16036bdb None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:59.081923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a13c8-8c12-486c-b4e7-b9d1ab44bc8f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:59.118873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b6f44-3b0a-40e2-8732-5e628556ab1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:59.119576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b6f44-3b0a-40e2-8732-5e628556ab1f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:59.119943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b6f44-3b0a-40e2-8732-5e628556ab1f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:59.120342 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b6f44-3b0a-40e2-8732-5e628556ab1f None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:59.121235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b6f44-3b0a-40e2-8732-5e628556ab1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:59.121640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b6f44-3b0a-40e2-8732-5e628556ab1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:59.123201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b6f44-3b0a-40e2-8732-5e628556ab1f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:59.136112 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 372/738] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:59.137410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b6f44-3b0a-40e2-8732-5e628556ab1f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:59.154744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:59.164368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b403dc9-8635-4a01-aad1-f24b16036bdb None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:59.199773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1aa2cd-061d-4f81-b76b-dc4b99b128c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:59.201365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1aa2cd-061d-4f81-b76b-dc4b99b128c8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:59.201816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1aa2cd-061d-4f81-b76b-dc4b99b128c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:59.202472 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1aa2cd-061d-4f81-b76b-dc4b99b128c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:59.211817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1aa2cd-061d-4f81-b76b-dc4b99b128c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:59.214345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1aa2cd-061d-4f81-b76b-dc4b99b128c8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-6f1aa2cd-061d-4f81-b76b-dc4b99b128c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:59.230336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:59.231532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:59.232644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:59.232644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:59.232644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:59.232644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:59.232644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:59.232644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:59.232644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:59.232644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:59.232644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 04:15:59.232644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:59.238995 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 367/739] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:15:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:59.238995 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1aa2cd-061d-4f81-b76b-dc4b99b128c8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:59.263540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b6f44-3b0a-40e2-8732-5e628556ab1f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:59.302227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:59.305131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570632e5-9957-4347-a0c0-9609126616cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:59.305780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570632e5-9957-4347-a0c0-9609126616cc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:59.306141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570632e5-9957-4347-a0c0-9609126616cc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:59.306412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570632e5-9957-4347-a0c0-9609126616cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:59.317775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570632e5-9957-4347-a0c0-9609126616cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-570632e5-9957-4347-a0c0-9609126616cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:15:59.323409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:59.324389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 373/740] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:15:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:59.325857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570632e5-9957-4347-a0c0-9609126616cc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:59.355569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1aa2cd-061d-4f81-b76b-dc4b99b128c8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:59.393004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b5dbd-8777-467a-b0fb-0d36d04f2fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:59.394412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b5dbd-8777-467a-b0fb-0d36d04f2fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:59.395600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b5dbd-8777-467a-b0fb-0d36d04f2fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:59.395600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b5dbd-8777-467a-b0fb-0d36d04f2fa4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:59.403029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b5dbd-8777-467a-b0fb-0d36d04f2fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:59.404479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b5dbd-8777-467a-b0fb-0d36d04f2fa4 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:59.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-090b5dbd-8777-467a-b0fb-0d36d04f2fa4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:15:59.415694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:15:59.416642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:15:59.418348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:15:59.418348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:15:59.418348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:15:59.418348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:15:59.418348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:15:59.418348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:15:59.418348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:15:59.418348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:15:59.418348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 04:15:59.418348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:15:59.418348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 368/741] 10.222.0.60 () {56 vars in 1117 bytes} [Sat Sep 17 04:15:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:15:59.418348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b5dbd-8777-467a-b0fb-0d36d04f2fa4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:59.440570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570632e5-9957-4347-a0c0-9609126616cc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:59.476958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8499b-f787-423c-a7f4-21376fe14dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:59.477603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8499b-f787-423c-a7f4-21376fe14dbb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:59.477912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8499b-f787-423c-a7f4-21376fe14dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:59.478185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8499b-f787-423c-a7f4-21376fe14dbb None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:59.478915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8499b-f787-423c-a7f4-21376fe14dbb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:59.479216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8499b-f787-423c-a7f4-21376fe14dbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:59.480660 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8499b-f787-423c-a7f4-21376fe14dbb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:59.498986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 374/742] 10.222.0.60 () {56 vars in 1135 bytes} [Sat Sep 17 04:15:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:59.499774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8499b-f787-423c-a7f4-21376fe14dbb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:59.533134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b5dbd-8777-467a-b0fb-0d36d04f2fa4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:59.551404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:59.571476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe778c3a-c0ad-450d-9d42-36d388a21273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:59.572084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe778c3a-c0ad-450d-9d42-36d388a21273 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:59.572355 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe778c3a-c0ad-450d-9d42-36d388a21273 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:59.572620 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe778c3a-c0ad-450d-9d42-36d388a21273 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:59.573304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe778c3a-c0ad-450d-9d42-36d388a21273 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:59.573597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe778c3a-c0ad-450d-9d42-36d388a21273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:59.575362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe778c3a-c0ad-450d-9d42-36d388a21273 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:59.584646 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 369/743] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:15:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:59.585404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe778c3a-c0ad-450d-9d42-36d388a21273 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:59.605419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8499b-f787-423c-a7f4-21376fe14dbb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:59.638351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:59.643542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad34f54-edd1-4feb-b1d4-d471b5af20c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:59.644558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad34f54-edd1-4feb-b1d4-d471b5af20c1 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:59.644830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad34f54-edd1-4feb-b1d4-d471b5af20c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:59.645128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad34f54-edd1-4feb-b1d4-d471b5af20c1 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:59.645838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad34f54-edd1-4feb-b1d4-d471b5af20c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:59.647202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad34f54-edd1-4feb-b1d4-d471b5af20c1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:59.669132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 375/744] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:15:59 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:15:59.669132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad34f54-edd1-4feb-b1d4-d471b5af20c1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:59.728535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe778c3a-c0ad-450d-9d42-36d388a21273 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:59.796831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0eb2a-dab2-4f34-8618-f93b7359db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:59.797914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0eb2a-dab2-4f34-8618-f93b7359db91 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:59.798423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0eb2a-dab2-4f34-8618-f93b7359db91 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:59.799026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0eb2a-dab2-4f34-8618-f93b7359db91 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:59.800090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0eb2a-dab2-4f34-8618-f93b7359db91 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:15:59.800604 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0eb2a-dab2-4f34-8618-f93b7359db91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:59.802488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0eb2a-dab2-4f34-8618-f93b7359db91 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:59.802873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad34f54-edd1-4feb-b1d4-d471b5af20c1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:15:59.823841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 370/745] 10.222.0.60 () {56 vars in 1149 bytes} [Sat Sep 17 04:15:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:15:59.825454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0eb2a-dab2-4f34-8618-f93b7359db91 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:15:59.853096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03b430-5913-4e79-a28b-ce342bed5b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:15:59.854206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03b430-5913-4e79-a28b-ce342bed5b49 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:15:59.854484 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03b430-5913-4e79-a28b-ce342bed5b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:15:59.854751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03b430-5913-4e79-a28b-ce342bed5b49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:15:59.864776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:59.927758 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:15:59.933898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03b430-5913-4e79-a28b-ce342bed5b49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:15:59.935593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03b430-5913-4e79-a28b-ce342bed5b49 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:15:59.958515 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0eb2a-dab2-4f34-8618-f93b7359db91 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:00.003352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e67ef8-0ecf-473d-a2c3-08e297c62e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:00.004639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e67ef8-0ecf-473d-a2c3-08e297c62e77 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:00.005458 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e67ef8-0ecf-473d-a2c3-08e297c62e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:00.005733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e67ef8-0ecf-473d-a2c3-08e297c62e77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:00.020448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e67ef8-0ecf-473d-a2c3-08e297c62e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:00.022940 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e67ef8-0ecf-473d-a2c3-08e297c62e77 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:00.042870 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 376/746] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:15:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:16:00.043806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03b430-5913-4e79-a28b-ce342bed5b49 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-32e67ef8-0ecf-473d-a2c3-08e297c62e77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:00.044945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:00.045971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:00.046852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:00.046852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:00.046852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:00.049888 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 371/747] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:00.053257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e67ef8-0ecf-473d-a2c3-08e297c62e77 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:00.139374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:00.149284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03b430-5913-4e79-a28b-ce342bed5b49 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:00.182053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:00.184391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e67ef8-0ecf-473d-a2c3-08e297c62e77 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:00.190230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6ee49-3ca9-4ece-9c3b-55a6e43821ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:00.190974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6ee49-3ca9-4ece-9c3b-55a6e43821ec None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:00.191308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6ee49-3ca9-4ece-9c3b-55a6e43821ec None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:00.191640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6ee49-3ca9-4ece-9c3b-55a6e43821ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:00.220488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad52f06-adea-459e-9a7a-c2f9d2abb4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:00.221150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad52f06-adea-459e-9a7a-c2f9d2abb4cd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:00.221428 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad52f06-adea-459e-9a7a-c2f9d2abb4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:00.221697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad52f06-adea-459e-9a7a-c2f9d2abb4cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:00.232996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad52f06-adea-459e-9a7a-c2f9d2abb4cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:00.234540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad52f06-adea-459e-9a7a-c2f9d2abb4cd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-7ad52f06-adea-459e-9a7a-c2f9d2abb4cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:00.252160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:00.253225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:00.254135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:00.254135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:00.254135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:00.254135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 372/748] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:00.254790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad52f06-adea-459e-9a7a-c2f9d2abb4cd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:00.285169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6ee49-3ca9-4ece-9c3b-55a6e43821ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:00.287291 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6ee49-3ca9-4ece-9c3b-55a6e43821ec None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:00.379134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad52f06-adea-459e-9a7a-c2f9d2abb4cd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:00.383263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:00.393374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 377/749] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:16:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:16:00.394277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6ee49-3ca9-4ece-9c3b-55a6e43821ec None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:00.429404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636d405-c725-4437-b9f0-be402794ef95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:00.430288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636d405-c725-4437-b9f0-be402794ef95 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:00.430728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636d405-c725-4437-b9f0-be402794ef95 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:00.431765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636d405-c725-4437-b9f0-be402794ef95 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:00.432479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636d405-c725-4437-b9f0-be402794ef95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:00.432729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636d405-c725-4437-b9f0-be402794ef95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:00.434151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636d405-c725-4437-b9f0-be402794ef95 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:00.437207 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:00.452508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 373/750] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:00.454343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636d405-c725-4437-b9f0-be402794ef95 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:00.502679 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6ee49-3ca9-4ece-9c3b-55a6e43821ec None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:00.539085 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1dad0-db5f-4685-95fc-a2c66cb9166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:00.540336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1dad0-db5f-4685-95fc-a2c66cb9166b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:00.541402 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1dad0-db5f-4685-95fc-a2c66cb9166b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:00.541873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1dad0-db5f-4685-95fc-a2c66cb9166b None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:00.542937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1dad0-db5f-4685-95fc-a2c66cb9166b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:00.543407 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1dad0-db5f-4685-95fc-a2c66cb9166b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:00.545919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1dad0-db5f-4685-95fc-a2c66cb9166b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:00.567751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 378/751] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:00.569416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1dad0-db5f-4685-95fc-a2c66cb9166b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:00.580840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636d405-c725-4437-b9f0-be402794ef95 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:00.618944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf325d-9ca2-4452-b7cc-e6050f4a5bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:00.619587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf325d-9ca2-4452-b7cc-e6050f4a5bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:00.619860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf325d-9ca2-4452-b7cc-e6050f4a5bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:00.620171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf325d-9ca2-4452-b7cc-e6050f4a5bd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:00.634075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf325d-9ca2-4452-b7cc-e6050f4a5bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:00.636574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf325d-9ca2-4452-b7cc-e6050f4a5bd8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:00.639354 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-4fdf325d-9ca2-4452-b7cc-e6050f4a5bd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:00.654819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:00.655765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:00.656640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:00.656640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:00.656640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:00.661039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 374/752] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:00.661591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf325d-9ca2-4452-b7cc-e6050f4a5bd8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:00.707491 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1dad0-db5f-4685-95fc-a2c66cb9166b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:00.717453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:00.754521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556fdcb-a7bf-4c5a-9473-fc0708080d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:00.755621 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556fdcb-a7bf-4c5a-9473-fc0708080d21 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:00.756040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556fdcb-a7bf-4c5a-9473-fc0708080d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:00.756546 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556fdcb-a7bf-4c5a-9473-fc0708080d21 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:00.766684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556fdcb-a7bf-4c5a-9473-fc0708080d21 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:00.769252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556fdcb-a7bf-4c5a-9473-fc0708080d21 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:00.774279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf325d-9ca2-4452-b7cc-e6050f4a5bd8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0556fdcb-a7bf-4c5a-9473-fc0708080d21 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:00.783385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:00.784523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:00.785705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:00.785705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:00.785705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:00.785705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:00.785705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:00.785705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:00.785705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:00.785705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:00.785705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:16:00.785705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:00.788624 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 379/753] 10.222.0.60 () {56 vars in 1093 bytes} [Sat Sep 17 04:16:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:00.789837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556fdcb-a7bf-4c5a-9473-fc0708080d21 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:00.814218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ab28d-0359-435b-8a4f-773e7c54ecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:00.815044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ab28d-0359-435b-8a4f-773e7c54ecfb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:00.815415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ab28d-0359-435b-8a4f-773e7c54ecfb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:00.815777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ab28d-0359-435b-8a4f-773e7c54ecfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:00.823126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ab28d-0359-435b-8a4f-773e7c54ecfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:00.825148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ab28d-0359-435b-8a4f-773e7c54ecfb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-ed6ab28d-0359-435b-8a4f-773e7c54ecfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:00.843505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:00.844504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:00.845407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:00.845407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:00.845407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:00.845407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:00.845407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:00.845407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:00.845407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:00.845407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:00.851462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 375/754] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:00.851462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ab28d-0359-435b-8a4f-773e7c54ecfb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:00.913736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556fdcb-a7bf-4c5a-9473-fc0708080d21 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:00.931079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:00.953183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181d03e-005e-47d0-8803-834b2d1f5c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:00.953888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181d03e-005e-47d0-8803-834b2d1f5c6e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:00.954226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181d03e-005e-47d0-8803-834b2d1f5c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:00.954557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181d03e-005e-47d0-8803-834b2d1f5c6e None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:00.955328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181d03e-005e-47d0-8803-834b2d1f5c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:00.955690 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181d03e-005e-47d0-8803-834b2d1f5c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:00.957160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181d03e-005e-47d0-8803-834b2d1f5c6e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:00.970072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 380/755] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:00.971072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181d03e-005e-47d0-8803-834b2d1f5c6e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:00.975735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ab28d-0359-435b-8a4f-773e7c54ecfb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:00.985418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:01.014053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b69ea0-fb89-4526-9b68-3aa12a17fa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:01.014670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b69ea0-fb89-4526-9b68-3aa12a17fa81 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:01.014948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b69ea0-fb89-4526-9b68-3aa12a17fa81 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:01.015328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b69ea0-fb89-4526-9b68-3aa12a17fa81 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:01.016169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b69ea0-fb89-4526-9b68-3aa12a17fa81 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:01.016471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b69ea0-fb89-4526-9b68-3aa12a17fa81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:01.017938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b69ea0-fb89-4526-9b68-3aa12a17fa81 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:01.040460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 376/756] 10.222.0.60 () {56 vars in 1133 bytes} [Sat Sep 17 04:16:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:01.040460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e063225-9e63-46cf-8412-99cdbbc6edf4 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:01.040460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e063225-9e63-46cf-8412-99cdbbc6edf4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:01.040460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e063225-9e63-46cf-8412-99cdbbc6edf4 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:01.040460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 377/757] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:01.040460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e063225-9e63-46cf-8412-99cdbbc6edf4 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:01.074139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181d03e-005e-47d0-8803-834b2d1f5c6e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:01.106837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddce7ce-3731-4dcc-9925-14c72d417916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:01.107438 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddce7ce-3731-4dcc-9925-14c72d417916 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:01.107708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddce7ce-3731-4dcc-9925-14c72d417916 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:01.107972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddce7ce-3731-4dcc-9925-14c72d417916 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:01.108672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddce7ce-3731-4dcc-9925-14c72d417916 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:01.108968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddce7ce-3731-4dcc-9925-14c72d417916 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:01.110414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddce7ce-3731-4dcc-9925-14c72d417916 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:01.122519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 381/758] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:01.123220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddce7ce-3731-4dcc-9925-14c72d417916 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:01.148152 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e063225-9e63-46cf-8412-99cdbbc6edf4 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:01.154709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:01.183687 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3032a-bfbf-44a4-8c38-a608ede3f040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:01.184565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3032a-bfbf-44a4-8c38-a608ede3f040 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:01.185020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3032a-bfbf-44a4-8c38-a608ede3f040 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:01.185486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3032a-bfbf-44a4-8c38-a608ede3f040 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:01.192255 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3032a-bfbf-44a4-8c38-a608ede3f040 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:01.194842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3032a-bfbf-44a4-8c38-a608ede3f040 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-41f3032a-bfbf-44a4-8c38-a608ede3f040 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:01.206244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:01.207146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:01.208062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:01.208062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:01.208062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:01.208062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:01.208062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:01.208062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:01.208062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:01.208062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:01.210103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 378/759] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:01.211393 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc38276-7bdd-48ea-86fc-45e145fe13b1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:01.211669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc38276-7bdd-48ea-86fc-45e145fe13b1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:01.211941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc38276-7bdd-48ea-86fc-45e145fe13b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:01.228062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddce7ce-3731-4dcc-9925-14c72d417916 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:01.229774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-bfc38276-7bdd-48ea-86fc-45e145fe13b1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:01.230950 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bfc38276-7bdd-48ea-86fc-45e145fe13b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:01.253733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:01.262713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8492a-bb1c-4e1a-af8a-5e8b415353ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:01.263308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8492a-bb1c-4e1a-af8a-5e8b415353ee None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:01.263583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8492a-bb1c-4e1a-af8a-5e8b415353ee None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:01.263846 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8492a-bb1c-4e1a-af8a-5e8b415353ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:01.274726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8492a-bb1c-4e1a-af8a-5e8b415353ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:01.276180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8492a-bb1c-4e1a-af8a-5e8b415353ee None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-29d8492a-bb1c-4e1a-af8a-5e8b415353ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:01.290249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:01.291221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:01.292146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:01.292146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:01.292146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:01.296277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 382/760] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:01.296912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8492a-bb1c-4e1a-af8a-5e8b415353ee None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:01.313218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 379/761] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:01.313218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc38276-7bdd-48ea-86fc-45e145fe13b1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:01.389259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:01.407320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8492a-bb1c-4e1a-af8a-5e8b415353ee None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:01.434302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc38276-7bdd-48ea-86fc-45e145fe13b1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:01.444002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba571c7-b28d-4fe7-bc6f-4c647a85f3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:01.444891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba571c7-b28d-4fe7-bc6f-4c647a85f3ae None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:01.445350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba571c7-b28d-4fe7-bc6f-4c647a85f3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:01.445761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba571c7-b28d-4fe7-bc6f-4c647a85f3ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:01.446699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba571c7-b28d-4fe7-bc6f-4c647a85f3ae None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=7aac7cd1742449328facf2902483f259) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:01.447168 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba571c7-b28d-4fe7-bc6f-4c647a85f3ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:01.448741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba571c7-b28d-4fe7-bc6f-4c647a85f3ae None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:01.461586 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 383/762] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:16:01 2022] GET /identity/v3/role_assignments?user.id=7aac7cd1742449328facf2902483f259&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:01.462847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba571c7-b28d-4fe7-bc6f-4c647a85f3ae None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:01.487956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4323f49-7c1d-43e0-9bce-4341af42ac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:01.488966 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4323f49-7c1d-43e0-9bce-4341af42ac80 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:01.489459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4323f49-7c1d-43e0-9bce-4341af42ac80 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:01.489943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4323f49-7c1d-43e0-9bce-4341af42ac80 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:01.490863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4323f49-7c1d-43e0-9bce-4341af42ac80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:01.492432 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4323f49-7c1d-43e0-9bce-4341af42ac80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:01.495463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4323f49-7c1d-43e0-9bce-4341af42ac80 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:01.525146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 380/763] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:01.538496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4323f49-7c1d-43e0-9bce-4341af42ac80 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:01.570260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:01.651566 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba571c7-b28d-4fe7-bc6f-4c647a85f3ae None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:01.691658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4323f49-7c1d-43e0-9bce-4341af42ac80 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:01.708534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115b5a5-78f9-4273-a2cc-e351f78ca708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:01.709358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115b5a5-78f9-4273-a2cc-e351f78ca708 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:01.709642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115b5a5-78f9-4273-a2cc-e351f78ca708 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:01.709912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115b5a5-78f9-4273-a2cc-e351f78ca708 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:01.710848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115b5a5-78f9-4273-a2cc-e351f78ca708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:01.711149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115b5a5-78f9-4273-a2cc-e351f78ca708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:01.712697 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115b5a5-78f9-4273-a2cc-e351f78ca708 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:01.727257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf5202-c8d4-4d37-a4ae-d981c55021bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:01.727774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 384/764] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:01.728144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf5202-c8d4-4d37-a4ae-d981c55021bf None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:01.728490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf5202-c8d4-4d37-a4ae-d981c55021bf None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:01.728596 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115b5a5-78f9-4273-a2cc-e351f78ca708 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:01.728839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf5202-c8d4-4d37-a4ae-d981c55021bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:01.729777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf5202-c8d4-4d37-a4ae-d981c55021bf None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=7c1a7d2677184c24ade81c397b7d09d1) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:01.730150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf5202-c8d4-4d37-a4ae-d981c55021bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:01.731662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf5202-c8d4-4d37-a4ae-d981c55021bf None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:01.741924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:01.744186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 381/765] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:16:01 2022] GET /identity/v3/role_assignments?user.id=7c1a7d2677184c24ade81c397b7d09d1&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:01.745611 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf5202-c8d4-4d37-a4ae-d981c55021bf None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:01.848404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115b5a5-78f9-4273-a2cc-e351f78ca708 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:01.887888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fba21-9ec8-49a0-b8b8-9d2a0f5d068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:01.889118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fba21-9ec8-49a0-b8b8-9d2a0f5d068e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:01.889402 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fba21-9ec8-49a0-b8b8-9d2a0f5d068e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:01.889669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fba21-9ec8-49a0-b8b8-9d2a0f5d068e None admin] PATH_INFO: `/v3/services/network` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:01.890268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fba21-9ec8-49a0-b8b8-9d2a0f5d068e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:01.891850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fba21-9ec8-49a0-b8b8-9d2a0f5d068e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-7b5fba21-9ec8-49a0-b8b8-9d2a0f5d068e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:01.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:01.903336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:01.904251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 385/766] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:01 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:01.912395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fba21-9ec8-49a0-b8b8-9d2a0f5d068e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:01.944743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:01.974983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf5202-c8d4-4d37-a4ae-d981c55021bf None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:01.993756 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:02.017206 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec70c6-676d-4c57-8eed-14774c6957a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:02.017859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec70c6-676d-4c57-8eed-14774c6957a4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:02.018137 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec70c6-676d-4c57-8eed-14774c6957a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:02.018409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec70c6-676d-4c57-8eed-14774c6957a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:02.026059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec70c6-676d-4c57-8eed-14774c6957a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:02.027510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec70c6-676d-4c57-8eed-14774c6957a4 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:02.032031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fba21-9ec8-49a0-b8b8-9d2a0f5d068e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-cbec70c6-676d-4c57-8eed-14774c6957a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:02.039232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:02.040148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:02.041582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:02.041582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:02.041582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:02.041582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:02.041582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:02.041582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:02.041582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:02.041582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:02.045668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 382/767] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:02.047111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22291e0c-5b13-49fb-87be-c8fcca97dfd2 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:02.047396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22291e0c-5b13-49fb-87be-c8fcca97dfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:02.047670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22291e0c-5b13-49fb-87be-c8fcca97dfd2 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:02.049205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 383/768] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:02.049911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22291e0c-5b13-49fb-87be-c8fcca97dfd2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:02.072209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4671b-fe02-49e7-83e5-ca7237713b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:02.073268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4671b-fe02-49e7-83e5-ca7237713b13 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:02.073881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4671b-fe02-49e7-83e5-ca7237713b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:02.074408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4671b-fe02-49e7-83e5-ca7237713b13 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:02.075304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4671b-fe02-49e7-83e5-ca7237713b13 None admin] RBAC: Adding query filter params (name=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:02.075806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4671b-fe02-49e7-83e5-ca7237713b13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:02.077539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4671b-fe02-49e7-83e5-ca7237713b13 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:02.082888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 386/769] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:16:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:02.084434 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a855daa-73b3-4569-99ab-6058885ca644 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:02.084948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a855daa-73b3-4569-99ab-6058885ca644 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:02.085467 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a855daa-73b3-4569-99ab-6058885ca644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:02.105200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6a855daa-73b3-4569-99ab-6058885ca644 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:02.106815 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6a855daa-73b3-4569-99ab-6058885ca644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:02.166711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22291e0c-5b13-49fb-87be-c8fcca97dfd2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:02.176767 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 387/770] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:02.178023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a855daa-73b3-4569-99ab-6058885ca644 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:02.222919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5121eea-876d-4614-a072-fb97f21887bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:02.226237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5121eea-876d-4614-a072-fb97f21887bd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:02.226722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5121eea-876d-4614-a072-fb97f21887bd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:02.227228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5121eea-876d-4614-a072-fb97f21887bd None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:02.228217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5121eea-876d-4614-a072-fb97f21887bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:02.229015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5121eea-876d-4614-a072-fb97f21887bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:02.231897 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5121eea-876d-4614-a072-fb97f21887bd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:02.250080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:02.255245 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 384/771] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:02.260998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5121eea-876d-4614-a072-fb97f21887bd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:02.292256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:02.351989 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a855daa-73b3-4569-99ab-6058885ca644 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:02.418918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9991b6-30dd-4c6b-8a1e-264d3919dd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:02.419539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9991b6-30dd-4c6b-8a1e-264d3919dd5f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:02.419816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9991b6-30dd-4c6b-8a1e-264d3919dd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:02.420081 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9991b6-30dd-4c6b-8a1e-264d3919dd5f None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:02.420730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9991b6-30dd-4c6b-8a1e-264d3919dd5f None admin] RBAC: Adding query filter params (type=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:02.421452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9991b6-30dd-4c6b-8a1e-264d3919dd5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:02.423734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9991b6-30dd-4c6b-8a1e-264d3919dd5f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:02.434054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 388/772] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:16:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:02.439463 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9991b6-30dd-4c6b-8a1e-264d3919dd5f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:02.456321 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5121eea-876d-4614-a072-fb97f21887bd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:02.511546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15664e58-0f56-481b-8a49-7460f0bc92d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:02.512141 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15664e58-0f56-481b-8a49-7460f0bc92d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:02.512423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15664e58-0f56-481b-8a49-7460f0bc92d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:02.512701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15664e58-0f56-481b-8a49-7460f0bc92d0 None admin] PATH_INFO: `/v3/projects/58466195aca94b188d82c1b3a647e767/users/d3272d57d5d24e5a990a6d7c86e6d4bc/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:02.515081 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15664e58-0f56-481b-8a49-7460f0bc92d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=58466195aca94b188d82c1b3a647e767, user_id=d3272d57d5d24e5a990a6d7c86e6d4bc, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:02.516784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15664e58-0f56-481b-8a49-7460f0bc92d0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:02.540080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 385/773] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:16:02 2022] PUT /identity/v3/projects/58466195aca94b188d82c1b3a647e767/users/d3272d57d5d24e5a990a6d7c86e6d4bc/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:16:02.540080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15664e58-0f56-481b-8a49-7460f0bc92d0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:02.592194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9991b6-30dd-4c6b-8a1e-264d3919dd5f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:02.608785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:02.680787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15664e58-0f56-481b-8a49-7460f0bc92d0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:02.682668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:02.704440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d815c3-44d5-45da-8510-c4c7ac6ba52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:02.705304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d815c3-44d5-45da-8510-c4c7ac6ba52c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:02.705682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d815c3-44d5-45da-8510-c4c7ac6ba52c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:02.706047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d815c3-44d5-45da-8510-c4c7ac6ba52c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:02.706905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d815c3-44d5-45da-8510-c4c7ac6ba52c None admin] RBAC: Adding query filter params (interface=public, service_id=335d57570ba74c7f9f33d624bae0fca4, region_id=RegionOne) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:02.707419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d815c3-44d5-45da-8510-c4c7ac6ba52c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:02.709121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d815c3-44d5-45da-8510-c4c7ac6ba52c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:02.722506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807abe57-ef3a-44c4-9c23-aef7481cc42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:02.723755 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807abe57-ef3a-44c4-9c23-aef7481cc42c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:02.724180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807abe57-ef3a-44c4-9c23-aef7481cc42c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:02.724592 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807abe57-ef3a-44c4-9c23-aef7481cc42c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:02.725914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807abe57-ef3a-44c4-9c23-aef7481cc42c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:02.727910 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807abe57-ef3a-44c4-9c23-aef7481cc42c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:02.730272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 389/774] 10.222.0.60 () {56 vars in 1251 bytes} [Sat Sep 17 04:16:02 2022] GET /identity/v3/endpoints?service_id=335d57570ba74c7f9f33d624bae0fca4&interface=public®ion_id=RegionOne => generated 194 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:02.736187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d815c3-44d5-45da-8510-c4c7ac6ba52c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-807abe57-ef3a-44c4-9c23-aef7481cc42c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:02.738858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:02.739859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 04:16:02.740714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:02.744373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 386/775] 10.222.0.60 () {56 vars in 1129 bytes} [Sat Sep 17 04:16:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:02.756343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807abe57-ef3a-44c4-9c23-aef7481cc42c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:02.863284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d815c3-44d5-45da-8510-c4c7ac6ba52c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:02.900535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1459b-4527-4196-bac6-c8b7efa9bcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:02.901582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1459b-4527-4196-bac6-c8b7efa9bcde None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:02.902059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1459b-4527-4196-bac6-c8b7efa9bcde None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:02.902544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1459b-4527-4196-bac6-c8b7efa9bcde None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:02.903346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1459b-4527-4196-bac6-c8b7efa9bcde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:02.905003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1459b-4527-4196-bac6-c8b7efa9bcde None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:02.911184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 390/776] 10.222.0.60 () {56 vars in 1087 bytes} [Sat Sep 17 04:16:02 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:02.975591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807abe57-ef3a-44c4-9c23-aef7481cc42c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:03.003632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:03.015599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a57639-a61c-4169-8b54-26b0190d61d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:03.016235 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a57639-a61c-4169-8b54-26b0190d61d6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:03.016515 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a57639-a61c-4169-8b54-26b0190d61d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:03.016788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a57639-a61c-4169-8b54-26b0190d61d6 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:03.017469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a57639-a61c-4169-8b54-26b0190d61d6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:03.017771 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a57639-a61c-4169-8b54-26b0190d61d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:03.019181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a57639-a61c-4169-8b54-26b0190d61d6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:03.024895 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 387/777] 10.222.0.60 () {56 vars in 1124 bytes} [Sat Sep 17 04:16:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:03.030583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1459b-4527-4196-bac6-c8b7efa9bcde None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:03.109346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:03.155595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1459b-4527-4196-bac6-c8b7efa9bcde None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:03.192847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdda2d-d960-4c51-a089-33280a565312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:03.193866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdda2d-d960-4c51-a089-33280a565312 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:03.194287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdda2d-d960-4c51-a089-33280a565312 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:03.194700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdda2d-d960-4c51-a089-33280a565312 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:03.195501 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcdda2d-d960-4c51-a089-33280a565312 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:03.195945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcdda2d-d960-4c51-a089-33280a565312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:03.197564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcdda2d-d960-4c51-a089-33280a565312 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:03.203360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 391/778] 10.222.0.60 () {56 vars in 1124 bytes} [Sat Sep 17 04:16:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:03.210304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a57639-a61c-4169-8b54-26b0190d61d6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:03.323290 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a57639-a61c-4169-8b54-26b0190d61d6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:03.364651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fe0ff-f84a-45d8-8e29-d3a51a5222cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:03.365317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fe0ff-f84a-45d8-8e29-d3a51a5222cf None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:03.365596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fe0ff-f84a-45d8-8e29-d3a51a5222cf None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:03.365868 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fe0ff-f84a-45d8-8e29-d3a51a5222cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:03.366946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fe0ff-f84a-45d8-8e29-d3a51a5222cf None admin] RBAC: Adding query filter params (interface=public, service_id=11824409d7564befae62c482e2329333, region_id=RegionOne) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:03.367255 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fe0ff-f84a-45d8-8e29-d3a51a5222cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:03.368684 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fe0ff-f84a-45d8-8e29-d3a51a5222cf None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:03.379452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 388/779] 10.222.0.60 () {56 vars in 1251 bytes} [Sat Sep 17 04:16:03 2022] GET /identity/v3/endpoints?service_id=11824409d7564befae62c482e2329333&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:03.385249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdda2d-d960-4c51-a089-33280a565312 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:03.511256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdda2d-d960-4c51-a089-33280a565312 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:03.534930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:03.548340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd20d1-7e1e-4ce0-a981-888e7514b21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:03.549351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd20d1-7e1e-4ce0-a981-888e7514b21e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:03.549833 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd20d1-7e1e-4ce0-a981-888e7514b21e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:03.550304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd20d1-7e1e-4ce0-a981-888e7514b21e None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:03.551110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd20d1-7e1e-4ce0-a981-888e7514b21e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:03.552700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd20d1-7e1e-4ce0-a981-888e7514b21e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:03.558740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 392/780] 10.222.0.60 () {56 vars in 1087 bytes} [Sat Sep 17 04:16:03 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:04.069261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c94d5e-8440-4968-8db9-4a46778741a5 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:04.069843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c94d5e-8440-4968-8db9-4a46778741a5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:04.069843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c94d5e-8440-4968-8db9-4a46778741a5 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:04.071610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 389/781] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:04.077555 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d099b7-d19f-4fb4-bb2f-a12337c37ffd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:04.078092 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d099b7-d19f-4fb4-bb2f-a12337c37ffd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:04.078092 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d099b7-d19f-4fb4-bb2f-a12337c37ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:04.109965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-36d099b7-d19f-4fb4-bb2f-a12337c37ffd None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:04.111246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-36d099b7-d19f-4fb4-bb2f-a12337c37ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:04.201442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec757e4-6645-4a3d-8e2b-157a1caf5b74 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:04.201798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec757e4-6645-4a3d-8e2b-157a1caf5b74 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:04.201996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec757e4-6645-4a3d-8e2b-157a1caf5b74 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:04.203807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 390/782] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:04.204606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 393/783] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:04.208977 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a59be1-8782-4c93-8416-56cfd9bf64d9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:04.209287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a59be1-8782-4c93-8416-56cfd9bf64d9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:04.209566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a59be1-8782-4c93-8416-56cfd9bf64d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:04.245454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-35a59be1-8782-4c93-8416-56cfd9bf64d9 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:04.246664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-35a59be1-8782-4c93-8416-56cfd9bf64d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:04.309968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:04.332996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 391/784] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:04.486382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d099b7-d19f-4fb4-bb2f-a12337c37ffd None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:04.586351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d099b7-d19f-4fb4-bb2f-a12337c37ffd None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:04.603719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:04.622595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcddd8-4a42-4fb8-a8b3-39b0102bde38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:04.623583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcddd8-4a42-4fb8-a8b3-39b0102bde38 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:04.624059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcddd8-4a42-4fb8-a8b3-39b0102bde38 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:04.624528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcddd8-4a42-4fb8-a8b3-39b0102bde38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:04.635219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bcddd8-4a42-4fb8-a8b3-39b0102bde38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f9bcddd8-4a42-4fb8-a8b3-39b0102bde38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:04.640325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:04.641283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:04.642133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:04.642133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:04.642133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:04.642133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:04.642133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:04.642133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:04.642133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:04.642133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:04.642133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:04.642133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:04.642133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:16:04.642133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:04.644712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 394/785] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:16:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:04.654236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a59be1-8782-4c93-8416-56cfd9bf64d9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:04.682211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcddd8-4a42-4fb8-a8b3-39b0102bde38 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:04.821061 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcddd8-4a42-4fb8-a8b3-39b0102bde38 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:04.824367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a59be1-8782-4c93-8416-56cfd9bf64d9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:04.856392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e75af-0344-4f0a-9fc1-95365b316a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:04.856954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e75af-0344-4f0a-9fc1-95365b316a96 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:04.857252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e75af-0344-4f0a-9fc1-95365b316a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:04.857527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e75af-0344-4f0a-9fc1-95365b316a96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:04.868634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e75af-0344-4f0a-9fc1-95365b316a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ab2e75af-0344-4f0a-9fc1-95365b316a96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:04.873603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:04.874542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:04.875394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 395/786] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:16:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:04.881525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e75af-0344-4f0a-9fc1-95365b316a96 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:04.894859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dd37f-61bf-401e-8d23-6add7f78f853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:04.895480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dd37f-61bf-401e-8d23-6add7f78f853 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:04.895751 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dd37f-61bf-401e-8d23-6add7f78f853 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:04.896017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dd37f-61bf-401e-8d23-6add7f78f853 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:04.896690 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6dd37f-61bf-401e-8d23-6add7f78f853 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:04.896985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6dd37f-61bf-401e-8d23-6add7f78f853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:04.898383 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6dd37f-61bf-401e-8d23-6add7f78f853 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:04.913615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 392/787] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:04.917058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dd37f-61bf-401e-8d23-6add7f78f853 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:04.945839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:05.000172 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:05.052069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e75af-0344-4f0a-9fc1-95365b316a96 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:05.090613 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a725ad-31db-4df0-b865-4f6ac2439754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:05.091869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a725ad-31db-4df0-b865-4f6ac2439754 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:05.092145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a725ad-31db-4df0-b865-4f6ac2439754 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:05.092419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a725ad-31db-4df0-b865-4f6ac2439754 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:05.093094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a725ad-31db-4df0-b865-4f6ac2439754 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:05.093402 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a725ad-31db-4df0-b865-4f6ac2439754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:05.094786 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a725ad-31db-4df0-b865-4f6ac2439754 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:05.101757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dd37f-61bf-401e-8d23-6add7f78f853 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:05.114803 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 396/788] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:16:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:05.126072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a725ad-31db-4df0-b865-4f6ac2439754 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:05.144987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69c0f2-cb8b-4882-aaf2-76941813beac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:05.145761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69c0f2-cb8b-4882-aaf2-76941813beac None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:05.146039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69c0f2-cb8b-4882-aaf2-76941813beac None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:05.146304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69c0f2-cb8b-4882-aaf2-76941813beac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:05.157062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69c0f2-cb8b-4882-aaf2-76941813beac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:05.158849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69c0f2-cb8b-4882-aaf2-76941813beac None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-fc69c0f2-cb8b-4882-aaf2-76941813beac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:05.172972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:05.174006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:05.174925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:05.174925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:05.174925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:05.174925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 393/789] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:05.179817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69c0f2-cb8b-4882-aaf2-76941813beac None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:05.293109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69c0f2-cb8b-4882-aaf2-76941813beac None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:05.293449 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:05.328659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d4d20-c058-4c8e-8956-28d0f81dc545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:05.329339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d4d20-c058-4c8e-8956-28d0f81dc545 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:05.329620 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d4d20-c058-4c8e-8956-28d0f81dc545 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:05.329892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d4d20-c058-4c8e-8956-28d0f81dc545 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:05.330568 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d4d20-c058-4c8e-8956-28d0f81dc545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:05.330833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d4d20-c058-4c8e-8956-28d0f81dc545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:05.337104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:05.337308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d4d20-c058-4c8e-8956-28d0f81dc545 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:05.341528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a725ad-31db-4df0-b865-4f6ac2439754 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:05.357138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 394/790] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:05.358510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d4d20-c058-4c8e-8956-28d0f81dc545 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:05.388554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d59e37-274c-4c4f-ab77-7b3b3b75a39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:05.389198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d59e37-274c-4c4f-ab77-7b3b3b75a39e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:05.389477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d59e37-274c-4c4f-ab77-7b3b3b75a39e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:05.389788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d59e37-274c-4c4f-ab77-7b3b3b75a39e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:05.401633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d59e37-274c-4c4f-ab77-7b3b3b75a39e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:05.403289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d59e37-274c-4c4f-ab77-7b3b3b75a39e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a6d59e37-274c-4c4f-ab77-7b3b3b75a39e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:05.418585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:05.419618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:05.420515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:05.420515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:05.420515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:05.420515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 397/791] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:05.425188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07993311-1718-493c-b3c0-48785f2a2dfa None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:05.425188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07993311-1718-493c-b3c0-48785f2a2dfa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:05.425188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07993311-1718-493c-b3c0-48785f2a2dfa None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:05.427748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 398/792] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:05.428710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07993311-1718-493c-b3c0-48785f2a2dfa None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:05.548455 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07993311-1718-493c-b3c0-48785f2a2dfa None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:05.591097 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f07cac-844b-49e9-88b8-8e01fa1ec8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:05.591877 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f07cac-844b-49e9-88b8-8e01fa1ec8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:05.592143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f07cac-844b-49e9-88b8-8e01fa1ec8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:05.592779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f07cac-844b-49e9-88b8-8e01fa1ec8b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:05.593780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f07cac-844b-49e9-88b8-8e01fa1ec8b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:05.594087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f07cac-844b-49e9-88b8-8e01fa1ec8b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:05.596024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f07cac-844b-49e9-88b8-8e01fa1ec8b0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:05.612506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d4d20-c058-4c8e-8956-28d0f81dc545 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:05.616135 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 399/793] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:05.618797 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4130a-1060-4d92-9c42-2cd89ebe66d2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:05.619080 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4130a-1060-4d92-9c42-2cd89ebe66d2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:05.619353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4130a-1060-4d92-9c42-2cd89ebe66d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:05.646825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-33f4130a-1060-4d92-9c42-2cd89ebe66d2 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:05.647628 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-33f4130a-1060-4d92-9c42-2cd89ebe66d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:05.656763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:05.660046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6f783-e97c-4f54-81a6-4cb9d2cfdc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:05.660972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6f783-e97c-4f54-81a6-4cb9d2cfdc27 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:05.661518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6f783-e97c-4f54-81a6-4cb9d2cfdc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:05.662036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6f783-e97c-4f54-81a6-4cb9d2cfdc27 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:05.673780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6f783-e97c-4f54-81a6-4cb9d2cfdc27 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:05.675507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6f783-e97c-4f54-81a6-4cb9d2cfdc27 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-bbc6f783-e97c-4f54-81a6-4cb9d2cfdc27 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:05.687913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:05.689044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:05.690018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:05.690018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:05.690018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:05.690018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:05.690018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:05.690018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:05.690018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:05.690018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:05.690018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 04:16:05.690018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:05.692668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 395/794] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:16:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:05.693759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6f783-e97c-4f54-81a6-4cb9d2cfdc27 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:05.713852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:05.741584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 400/795] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:05.742691 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4130a-1060-4d92-9c42-2cd89ebe66d2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:05.804668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6f783-e97c-4f54-81a6-4cb9d2cfdc27 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:05.844169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae4edd-a0c0-4584-988d-06bf5418f293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:05.845196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4edd-a0c0-4584-988d-06bf5418f293 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:05.845654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4edd-a0c0-4584-988d-06bf5418f293 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:05.846098 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4edd-a0c0-4584-988d-06bf5418f293 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:05.855551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae4edd-a0c0-4584-988d-06bf5418f293 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:05.857403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae4edd-a0c0-4584-988d-06bf5418f293 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-f0ae4edd-a0c0-4584-988d-06bf5418f293 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:05.870850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:05.871878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:05.872806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:05.872806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:05.872806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:05.872806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:05.872806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:05.872806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:05.872806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:05.872806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:05.872806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 04:16:05.872806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:05.875289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 396/796] 10.222.0.60 () {56 vars in 1117 bytes} [Sat Sep 17 04:16:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:05.876729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f642f-a26b-4675-a507-263e4722c142 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:05.877185 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f642f-a26b-4675-a507-263e4722c142 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:05.877594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f642f-a26b-4675-a507-263e4722c142 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:05.879197 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 397/797] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:05.880380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f642f-a26b-4675-a507-263e4722c142 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:05.966567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:05.982884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4130a-1060-4d92-9c42-2cd89ebe66d2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:05.993857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f642f-a26b-4675-a507-263e4722c142 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:06.017022 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b686b-22ee-4924-a446-e3b41ef18422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:06.017650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b686b-22ee-4924-a446-e3b41ef18422 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.017920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b686b-22ee-4924-a446-e3b41ef18422 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.018186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b686b-22ee-4924-a446-e3b41ef18422 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.018986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b686b-22ee-4924-a446-e3b41ef18422 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:06.019192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b686b-22ee-4924-a446-e3b41ef18422 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:06.020686 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b686b-22ee-4924-a446-e3b41ef18422 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:06.029975 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:06.031409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058cde8-b58a-4b38-b767-afd2301626a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:06.032110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058cde8-b58a-4b38-b767-afd2301626a4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.032420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058cde8-b58a-4b38-b767-afd2301626a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.032639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058cde8-b58a-4b38-b767-afd2301626a4 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.033441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058cde8-b58a-4b38-b767-afd2301626a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:06.033743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058cde8-b58a-4b38-b767-afd2301626a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:06.035167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058cde8-b58a-4b38-b767-afd2301626a4 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:06.044064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 401/798] 10.222.0.60 () {56 vars in 1135 bytes} [Sat Sep 17 04:16:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:06.044064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682317d4-9a84-4a78-8074-55a6ccb509c4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.044064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682317d4-9a84-4a78-8074-55a6ccb509c4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.044064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682317d4-9a84-4a78-8074-55a6ccb509c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.057202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 398/799] 10.222.0.60 () {56 vars in 1149 bytes} [Sat Sep 17 04:16:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:06.057202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058cde8-b58a-4b38-b767-afd2301626a4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:06.060260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-682317d4-9a84-4a78-8074-55a6ccb509c4 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:06.061760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-682317d4-9a84-4a78-8074-55a6ccb509c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:06.134941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 402/800] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:06.137561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682317d4-9a84-4a78-8074-55a6ccb509c4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:06.160761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058cde8-b58a-4b38-b767-afd2301626a4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:06.198099 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e8bd6-0ec1-4cb4-8065-c9d3c6991b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:06.198738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e8bd6-0ec1-4cb4-8065-c9d3c6991b1c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.199014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e8bd6-0ec1-4cb4-8065-c9d3c6991b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.199374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e8bd6-0ec1-4cb4-8065-c9d3c6991b1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.201138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:06.212090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e8bd6-0ec1-4cb4-8065-c9d3c6991b1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c02e8bd6-0ec1-4cb4-8065-c9d3c6991b1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:06.217782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:06.219223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:06.220071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:06.220071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:06.220071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:06.220071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:06.220071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:06.220071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:06.220071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:06.220071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:06.220071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:06.220071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:06.220071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:16:06.220071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:06.224073 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 399/801] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:16:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:06.225234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e8bd6-0ec1-4cb4-8065-c9d3c6991b1c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:06.273008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682317d4-9a84-4a78-8074-55a6ccb509c4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:06.301911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:06.311639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85521064-06f9-433e-9e8f-67b1dcd7cee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:06.312731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85521064-06f9-433e-9e8f-67b1dcd7cee6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.313254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85521064-06f9-433e-9e8f-67b1dcd7cee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.313815 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85521064-06f9-433e-9e8f-67b1dcd7cee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.325508 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85521064-06f9-433e-9e8f-67b1dcd7cee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:06.327639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85521064-06f9-433e-9e8f-67b1dcd7cee6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:06.335830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e8bd6-0ec1-4cb4-8065-c9d3c6991b1c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-85521064-06f9-433e-9e8f-67b1dcd7cee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:06.343964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:06.344925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:06.345852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:06.345852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:06.345852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:06.348001 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 403/802] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:06.349068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85521064-06f9-433e-9e8f-67b1dcd7cee6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:06.379855 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc798a4-2efb-4ede-9a04-031afdcf8746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:06.380492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc798a4-2efb-4ede-9a04-031afdcf8746 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.380766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc798a4-2efb-4ede-9a04-031afdcf8746 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.381223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc798a4-2efb-4ede-9a04-031afdcf8746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.391584 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc798a4-2efb-4ede-9a04-031afdcf8746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:06.393013 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc798a4-2efb-4ede-9a04-031afdcf8746 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-4dc798a4-2efb-4ede-9a04-031afdcf8746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:06.408105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:06.409125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:06.410644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:06.410644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:06.410644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:06.410644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 400/803] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:06.410644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ebde6-667f-4ca2-8788-2ee70329c9b7 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.410984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ebde6-667f-4ca2-8788-2ee70329c9b7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.410984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ebde6-667f-4ca2-8788-2ee70329c9b7 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.412221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 401/804] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:06.412908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ebde6-667f-4ca2-8788-2ee70329c9b7 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:06.454023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85521064-06f9-433e-9e8f-67b1dcd7cee6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:06.458075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:06.489710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c17dc9-ad0e-4b8f-ad3c-b61dd834e409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:06.490241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c17dc9-ad0e-4b8f-ad3c-b61dd834e409 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.490507 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c17dc9-ad0e-4b8f-ad3c-b61dd834e409 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.490934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c17dc9-ad0e-4b8f-ad3c-b61dd834e409 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.492486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c17dc9-ad0e-4b8f-ad3c-b61dd834e409 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:06.493014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c17dc9-ad0e-4b8f-ad3c-b61dd834e409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:06.495546 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c17dc9-ad0e-4b8f-ad3c-b61dd834e409 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:06.509569 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 404/805] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:16:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:06.509569 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c17dc9-ad0e-4b8f-ad3c-b61dd834e409 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:06.517969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ebde6-667f-4ca2-8788-2ee70329c9b7 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:06.547496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:06.550388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6972c7c-59d7-415b-98f8-5a3c484f1617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:06.550966 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6972c7c-59d7-415b-98f8-5a3c484f1617 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.551241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6972c7c-59d7-415b-98f8-5a3c484f1617 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.551507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6972c7c-59d7-415b-98f8-5a3c484f1617 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.552146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6972c7c-59d7-415b-98f8-5a3c484f1617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:06.552445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6972c7c-59d7-415b-98f8-5a3c484f1617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:06.553829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6972c7c-59d7-415b-98f8-5a3c484f1617 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:06.565497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 402/806] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:06.566164 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6972c7c-59d7-415b-98f8-5a3c484f1617 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:06.608318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c17dc9-ad0e-4b8f-ad3c-b61dd834e409 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:06.641418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa684f-9122-42d6-ad94-82ffcad115e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:06.642038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa684f-9122-42d6-ad94-82ffcad115e2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.642313 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa684f-9122-42d6-ad94-82ffcad115e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.642579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa684f-9122-42d6-ad94-82ffcad115e2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.643156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa684f-9122-42d6-ad94-82ffcad115e2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:06.644510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa684f-9122-42d6-ad94-82ffcad115e2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-eefa684f-9122-42d6-ad94-82ffcad115e2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:06.653120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:06.654303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:06.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 405/807] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:06 2022] GET /identity/v3/services/network => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:06.656002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4f66a-d7c7-48b2-8853-f437ec4b5948 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.656002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4f66a-d7c7-48b2-8853-f437ec4b5948 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.656002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4f66a-d7c7-48b2-8853-f437ec4b5948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.667503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6972c7c-59d7-415b-98f8-5a3c484f1617 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:06.675116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:06.675270 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a1b4f66a-d7c7-48b2-8853-f437ec4b5948 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:06.676352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a1b4f66a-d7c7-48b2-8853-f437ec4b5948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:06.700644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df7cc6-5201-4a45-88c0-bcf6b920c5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:06.701337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df7cc6-5201-4a45-88c0-bcf6b920c5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.701612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df7cc6-5201-4a45-88c0-bcf6b920c5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.701872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df7cc6-5201-4a45-88c0-bcf6b920c5f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.702505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df7cc6-5201-4a45-88c0-bcf6b920c5f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:06.702810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df7cc6-5201-4a45-88c0-bcf6b920c5f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:06.704154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df7cc6-5201-4a45-88c0-bcf6b920c5f6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:06.715077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 403/808] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:06.715846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df7cc6-5201-4a45-88c0-bcf6b920c5f6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:06.743252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 406/809] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:06.744166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4f66a-d7c7-48b2-8853-f437ec4b5948 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:06.789723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:06.818269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df7cc6-5201-4a45-88c0-bcf6b920c5f6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:06.853230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08555399-05c0-45d9-976f-26fac60111c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:06.853948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08555399-05c0-45d9-976f-26fac60111c7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.854220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08555399-05c0-45d9-976f-26fac60111c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.854485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08555399-05c0-45d9-976f-26fac60111c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.858316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4f66a-d7c7-48b2-8853-f437ec4b5948 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:06.864708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08555399-05c0-45d9-976f-26fac60111c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:06.866544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08555399-05c0-45d9-976f-26fac60111c7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-08555399-05c0-45d9-976f-26fac60111c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:06.880784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:06.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:06.882738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:06.882738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:06.882738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:06.882738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 404/810] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:06.882738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08555399-05c0-45d9-976f-26fac60111c7 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:06.898615 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e828485-aa9a-4176-a641-48ea85e2db92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:06.899100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e828485-aa9a-4176-a641-48ea85e2db92 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:06.899376 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e828485-aa9a-4176-a641-48ea85e2db92 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:06.899704 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e828485-aa9a-4176-a641-48ea85e2db92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:06.902610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:06.907739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e828485-aa9a-4176-a641-48ea85e2db92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:06.909510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e828485-aa9a-4176-a641-48ea85e2db92 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8e828485-aa9a-4176-a641-48ea85e2db92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:06.920811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:06.921731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:06.923165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:06.923165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:06.923165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:06.923165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:06.923165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:06.923165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:06.923165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:06.923165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:06.923165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 407/811] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:06.923165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e828485-aa9a-4176-a641-48ea85e2db92 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:06.986147 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08555399-05c0-45d9-976f-26fac60111c7 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:07.019613 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de686cee-cc4c-4000-a499-5cf494affd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:07.020212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de686cee-cc4c-4000-a499-5cf494affd07 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:07.020486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de686cee-cc4c-4000-a499-5cf494affd07 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:07.020744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de686cee-cc4c-4000-a499-5cf494affd07 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:07.021395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de686cee-cc4c-4000-a499-5cf494affd07 None admin] RBAC: Adding query filter params (name=network) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:07.021689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de686cee-cc4c-4000-a499-5cf494affd07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:07.022169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:07.023043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de686cee-cc4c-4000-a499-5cf494affd07 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:07.032216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 405/812] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:16:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:07.032373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e828485-aa9a-4176-a641-48ea85e2db92 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:07.032483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de686cee-cc4c-4000-a499-5cf494affd07 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:07.060764 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fed2b4-090b-45ad-8091-2cbd0348d4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:07.061446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fed2b4-090b-45ad-8091-2cbd0348d4ec None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:07.061713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fed2b4-090b-45ad-8091-2cbd0348d4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:07.061979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fed2b4-090b-45ad-8091-2cbd0348d4ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:07.068324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fed2b4-090b-45ad-8091-2cbd0348d4ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:07.069804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fed2b4-090b-45ad-8091-2cbd0348d4ec None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-62fed2b4-090b-45ad-8091-2cbd0348d4ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:07.080619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:07.081650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:07.082540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:07.082540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:07.082540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:07.082540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:07.082540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:07.082540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:07.082540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:07.082540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:07.082540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 408/813] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:07.082540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fed2b4-090b-45ad-8091-2cbd0348d4ec None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:07.108527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:07.126483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de686cee-cc4c-4000-a499-5cf494affd07 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:07.158685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6deeb5-26d7-467a-bb5c-20256c67f415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:07.159253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6deeb5-26d7-467a-bb5c-20256c67f415 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:07.159525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6deeb5-26d7-467a-bb5c-20256c67f415 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:07.159791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6deeb5-26d7-467a-bb5c-20256c67f415 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:07.160429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6deeb5-26d7-467a-bb5c-20256c67f415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:07.160729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6deeb5-26d7-467a-bb5c-20256c67f415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:07.162114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6deeb5-26d7-467a-bb5c-20256c67f415 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:07.173474 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 406/814] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:07.174248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6deeb5-26d7-467a-bb5c-20256c67f415 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:07.187374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fed2b4-090b-45ad-8091-2cbd0348d4ec None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:07.221843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ead74-57e4-472f-aa97-01667267d4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:07.222442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ead74-57e4-472f-aa97-01667267d4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:07.222718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ead74-57e4-472f-aa97-01667267d4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:07.222987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ead74-57e4-472f-aa97-01667267d4c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:07.223677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ead74-57e4-472f-aa97-01667267d4c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:07.223976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ead74-57e4-472f-aa97-01667267d4c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:07.225429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ead74-57e4-472f-aa97-01667267d4c1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:07.237579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 409/815] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:07.238453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ead74-57e4-472f-aa97-01667267d4c1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:07.255267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:07.268789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6deeb5-26d7-467a-bb5c-20256c67f415 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:07.300598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e119b54-6f5d-48a1-8799-0de83b6dbe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:07.301186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e119b54-6f5d-48a1-8799-0de83b6dbe03 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:07.301552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e119b54-6f5d-48a1-8799-0de83b6dbe03 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:07.301872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e119b54-6f5d-48a1-8799-0de83b6dbe03 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:07.302909 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e119b54-6f5d-48a1-8799-0de83b6dbe03 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:07.304367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e119b54-6f5d-48a1-8799-0de83b6dbe03 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-5e119b54-6f5d-48a1-8799-0de83b6dbe03 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:07.313300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:07.314285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:07.315161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 407/816] 10.222.0.60 () {56 vars in 1129 bytes} [Sat Sep 17 04:16:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 140 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:07.315950 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e119b54-6f5d-48a1-8799-0de83b6dbe03 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:07.323821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:07.339654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ead74-57e4-472f-aa97-01667267d4c1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:07.373899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00895677-d364-4aaf-a716-78b182839242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:07.374504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00895677-d364-4aaf-a716-78b182839242 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:07.374779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00895677-d364-4aaf-a716-78b182839242 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:07.375048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00895677-d364-4aaf-a716-78b182839242 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:07.375675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00895677-d364-4aaf-a716-78b182839242 None admin] RBAC: Adding query filter params (type=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:07.375971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00895677-d364-4aaf-a716-78b182839242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:07.377373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00895677-d364-4aaf-a716-78b182839242 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:07.382561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 410/817] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:16:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:07.383338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00895677-d364-4aaf-a716-78b182839242 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:07.411760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e119b54-6f5d-48a1-8799-0de83b6dbe03 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:07.444610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc8e7e-c59d-4c64-9eab-afea9af673cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:07.445258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc8e7e-c59d-4c64-9eab-afea9af673cd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:07.445533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc8e7e-c59d-4c64-9eab-afea9af673cd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:07.445802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc8e7e-c59d-4c64-9eab-afea9af673cd None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:07.446569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc8e7e-c59d-4c64-9eab-afea9af673cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:07.446874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc8e7e-c59d-4c64-9eab-afea9af673cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:07.448307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc8e7e-c59d-4c64-9eab-afea9af673cd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:07.458976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:07.460562 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 408/818] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:07.461416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc8e7e-c59d-4c64-9eab-afea9af673cd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:07.479940 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00895677-d364-4aaf-a716-78b182839242 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:07.515915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9881c-e2ea-4d6a-b124-268e13ceb2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:07.516567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9881c-e2ea-4d6a-b124-268e13ceb2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:07.516845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9881c-e2ea-4d6a-b124-268e13ceb2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:07.517216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9881c-e2ea-4d6a-b124-268e13ceb2f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:07.524168 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9881c-e2ea-4d6a-b124-268e13ceb2f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:07.525680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9881c-e2ea-4d6a-b124-268e13ceb2f3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:07.527041 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-40e9881c-e2ea-4d6a-b124-268e13ceb2f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:07.537927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:07.538909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:07.545356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:07.545356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:07.545356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:07.545356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:07.545356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:07.545356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:07.545356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:07.545356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:07.545356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:16:07.545356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:07.545356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 411/819] 10.222.0.60 () {56 vars in 1093 bytes} [Sat Sep 17 04:16:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:07.545356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9881c-e2ea-4d6a-b124-268e13ceb2f3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:07.567536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc8e7e-c59d-4c64-9eab-afea9af673cd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:07.601497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eccbf-26ac-432b-9233-8d5d8046fcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:07.602188 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eccbf-26ac-432b-9233-8d5d8046fcef None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:07.602469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eccbf-26ac-432b-9233-8d5d8046fcef None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:07.602766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eccbf-26ac-432b-9233-8d5d8046fcef None admin] PATH_INFO: `/v3/projects/58466195aca94b188d82c1b3a647e767/users/7aac7cd1742449328facf2902483f259/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:07.604688 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3eccbf-26ac-432b-9233-8d5d8046fcef None admin] RBAC: Authorizing `identity:create_grant(project_id=58466195aca94b188d82c1b3a647e767, user_id=7aac7cd1742449328facf2902483f259, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:07.606238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3eccbf-26ac-432b-9233-8d5d8046fcef None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:07.621101 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 409/820] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:16:07 2022] PUT /identity/v3/projects/58466195aca94b188d82c1b3a647e767/users/7aac7cd1742449328facf2902483f259/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:16:07.621907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eccbf-26ac-432b-9233-8d5d8046fcef None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:07.652202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9881c-e2ea-4d6a-b124-268e13ceb2f3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:07.694314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e282f-6a52-4cf8-af63-7bddd06dc93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:07.695341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e282f-6a52-4cf8-af63-7bddd06dc93b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:07.695630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e282f-6a52-4cf8-af63-7bddd06dc93b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:07.696081 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e282f-6a52-4cf8-af63-7bddd06dc93b None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:07.696736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e282f-6a52-4cf8-af63-7bddd06dc93b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:07.697046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e282f-6a52-4cf8-af63-7bddd06dc93b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:07.698510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e282f-6a52-4cf8-af63-7bddd06dc93b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:07.703667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 412/821] 10.222.0.60 () {56 vars in 1124 bytes} [Sat Sep 17 04:16:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:07.704511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e282f-6a52-4cf8-af63-7bddd06dc93b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:07.709989 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:07.744589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eccbf-26ac-432b-9233-8d5d8046fcef None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:07.772523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:07.783065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45299e9-9cb1-41e0-ae6d-4fd77afd4c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:07.783694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45299e9-9cb1-41e0-ae6d-4fd77afd4c8b None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:07.784544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45299e9-9cb1-41e0-ae6d-4fd77afd4c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:07.784821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45299e9-9cb1-41e0-ae6d-4fd77afd4c8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:07.785551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45299e9-9cb1-41e0-ae6d-4fd77afd4c8b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:07.786949 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45299e9-9cb1-41e0-ae6d-4fd77afd4c8b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:07.817214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 410/822] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:16:07 2022] POST /identity/v3/endpoints => generated 337 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:16:07.817924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45299e9-9cb1-41e0-ae6d-4fd77afd4c8b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:07.825517 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e282f-6a52-4cf8-af63-7bddd06dc93b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:07.881128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7f37e-1a69-4831-a1ab-b9315df09a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:07.881832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7f37e-1a69-4831-a1ab-b9315df09a57 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:07.882109 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7f37e-1a69-4831-a1ab-b9315df09a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:07.883031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7f37e-1a69-4831-a1ab-b9315df09a57 None admin] PATH_INFO: `/v3/projects/58466195aca94b188d82c1b3a647e767/users/7c1a7d2677184c24ade81c397b7d09d1/roles/3eccbe0521ef44d88574b9acd9f988db` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:07.906005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7f37e-1a69-4831-a1ab-b9315df09a57 None admin] RBAC: Authorizing `identity:create_grant(project_id=58466195aca94b188d82c1b3a647e767, user_id=7c1a7d2677184c24ade81c397b7d09d1, role_id=3eccbe0521ef44d88574b9acd9f988db)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:07.908384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7f37e-1a69-4831-a1ab-b9315df09a57 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:07.928433 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 413/823] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:16:07 2022] PUT /identity/v3/projects/58466195aca94b188d82c1b3a647e767/users/7c1a7d2677184c24ade81c397b7d09d1/roles/3eccbe0521ef44d88574b9acd9f988db => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:16:07.929443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7f37e-1a69-4831-a1ab-b9315df09a57 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:07.963784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45299e9-9cb1-41e0-ae6d-4fd77afd4c8b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:08.001043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:08.042607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8d0cb-44c6-4ea4-9788-4c26286877bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:08.043445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8d0cb-44c6-4ea4-9788-4c26286877bc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:08.044183 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8d0cb-44c6-4ea4-9788-4c26286877bc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:08.044464 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8d0cb-44c6-4ea4-9788-4c26286877bc None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:08.045551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8d0cb-44c6-4ea4-9788-4c26286877bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:08.045862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8d0cb-44c6-4ea4-9788-4c26286877bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:08.047865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8d0cb-44c6-4ea4-9788-4c26286877bc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:08.070797 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7f37e-1a69-4831-a1ab-b9315df09a57 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:08.074887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 411/824] 10.222.0.60 () {56 vars in 1133 bytes} [Sat Sep 17 04:16:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:08.082231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8d0cb-44c6-4ea4-9788-4c26286877bc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:08.120861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b89d7-a0c5-485f-b057-5de7e540c08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:08.122030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b89d7-a0c5-485f-b057-5de7e540c08a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:08.122363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b89d7-a0c5-485f-b057-5de7e540c08a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:08.122701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b89d7-a0c5-485f-b057-5de7e540c08a None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:08.123408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b89d7-a0c5-485f-b057-5de7e540c08a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:08.123773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b89d7-a0c5-485f-b057-5de7e540c08a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:08.125292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b89d7-a0c5-485f-b057-5de7e540c08a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:08.135494 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:08.137319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 414/825] 10.222.0.60 () {56 vars in 1124 bytes} [Sat Sep 17 04:16:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:08.138931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b89d7-a0c5-485f-b057-5de7e540c08a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:08.227918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8d0cb-44c6-4ea4-9788-4c26286877bc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:08.264883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b89d7-a0c5-485f-b057-5de7e540c08a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:08.266610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e2b81-c325-4a32-bb93-1c293fece52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:08.267228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e2b81-c325-4a32-bb93-1c293fece52b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:08.267520 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e2b81-c325-4a32-bb93-1c293fece52b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:08.267827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e2b81-c325-4a32-bb93-1c293fece52b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:08.285068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:08.309748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6889f22-aaad-4509-9225-051b9ba043b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:08.310310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6889f22-aaad-4509-9225-051b9ba043b2 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:08.310582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6889f22-aaad-4509-9225-051b9ba043b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:08.310846 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6889f22-aaad-4509-9225-051b9ba043b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:08.311520 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6889f22-aaad-4509-9225-051b9ba043b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:08.312941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6889f22-aaad-4509-9225-051b9ba043b2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:08.345471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 415/826] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:16:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:16:08.355714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e2b81-c325-4a32-bb93-1c293fece52b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:08.357802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e2b81-c325-4a32-bb93-1c293fece52b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:08.411572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:08.476141 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 412/827] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:16:08 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:16:08.481611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6889f22-aaad-4509-9225-051b9ba043b2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:08.616056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6889f22-aaad-4509-9225-051b9ba043b2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:08.656428 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877c538-f2df-4daa-9068-764442855d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:08.657062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877c538-f2df-4daa-9068-764442855d31 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:08.661468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877c538-f2df-4daa-9068-764442855d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:08.661749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877c538-f2df-4daa-9068-764442855d31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:08.678306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877c538-f2df-4daa-9068-764442855d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:08.679774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877c538-f2df-4daa-9068-764442855d31 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-9877c538-f2df-4daa-9068-764442855d31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:08.694793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:08.695776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:08.696753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:08.696753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:08.696753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:08.696753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 416/828] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:08.701869 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e2b81-c325-4a32-bb93-1c293fece52b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:08.803864 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e2b81-c325-4a32-bb93-1c293fece52b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:08.838246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c419be-02dc-4e7c-9379-8aaa25390190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:08.838852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c419be-02dc-4e7c-9379-8aaa25390190 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:08.839125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c419be-02dc-4e7c-9379-8aaa25390190 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:08.839390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c419be-02dc-4e7c-9379-8aaa25390190 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:08.840024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c419be-02dc-4e7c-9379-8aaa25390190 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:08.840323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c419be-02dc-4e7c-9379-8aaa25390190 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:08.841713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c419be-02dc-4e7c-9379-8aaa25390190 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:08.849956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:08.853329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 413/829] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:08.858857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877c538-f2df-4daa-9068-764442855d31 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:08.884512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:08.979123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877c538-f2df-4daa-9068-764442855d31 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:09.013312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de35c7-9e0a-484c-ac8a-9da46a9dc501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:09.014286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de35c7-9e0a-484c-ac8a-9da46a9dc501 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:09.014763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de35c7-9e0a-484c-ac8a-9da46a9dc501 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:09.015253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de35c7-9e0a-484c-ac8a-9da46a9dc501 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:09.022189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de35c7-9e0a-484c-ac8a-9da46a9dc501 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:09.024104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de35c7-9e0a-484c-ac8a-9da46a9dc501 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-22de35c7-9e0a-484c-ac8a-9da46a9dc501 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:09.035498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:09.036403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:09.037377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:09.037377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:09.037377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:09.037377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:09.037377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:09.037377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:09.037377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:09.037377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:09.040010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 417/830] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:09.045844 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c419be-02dc-4e7c-9379-8aaa25390190 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:09.143319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c419be-02dc-4e7c-9379-8aaa25390190 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:09.175424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00497e7b-4c13-4084-837c-7e55f602af28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:09.176028 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00497e7b-4c13-4084-837c-7e55f602af28 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:09.176315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00497e7b-4c13-4084-837c-7e55f602af28 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:09.176571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00497e7b-4c13-4084-837c-7e55f602af28 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:09.177343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00497e7b-4c13-4084-837c-7e55f602af28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:09.177646 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00497e7b-4c13-4084-837c-7e55f602af28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:09.179117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00497e7b-4c13-4084-837c-7e55f602af28 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:09.191358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 414/831] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:09.197401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de35c7-9e0a-484c-ac8a-9da46a9dc501 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:09.283417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:09.306855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de35c7-9e0a-484c-ac8a-9da46a9dc501 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:09.343285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfaf03-2215-47a3-bd27-397990266b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:09.343893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfaf03-2215-47a3-bd27-397990266b86 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:09.344174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfaf03-2215-47a3-bd27-397990266b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:09.344449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfaf03-2215-47a3-bd27-397990266b86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:09.345205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfaf03-2215-47a3-bd27-397990266b86 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=d3272d57d5d24e5a990a6d7c86e6d4bc) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:09.345509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfaf03-2215-47a3-bd27-397990266b86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:09.346950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfaf03-2215-47a3-bd27-397990266b86 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:09.353457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 418/832] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:16:09 2022] GET /identity/v3/role_assignments?user.id=d3272d57d5d24e5a990a6d7c86e6d4bc&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:09.862074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c04a07-22d4-4a00-864b-0faf5278e42b None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:09.862639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c04a07-22d4-4a00-864b-0faf5278e42b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:09.862639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c04a07-22d4-4a00-864b-0faf5278e42b None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:09.864091 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 415/833] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:09.870037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0c7ff-074a-4b3f-aa05-1d397f87867b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:09.870551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0c7ff-074a-4b3f-aa05-1d397f87867b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:09.871039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0c7ff-074a-4b3f-aa05-1d397f87867b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:09.889975 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f4e0c7ff-074a-4b3f-aa05-1d397f87867b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:09.891465 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f4e0c7ff-074a-4b3f-aa05-1d397f87867b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:09.963847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 419/834] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:16:10.230222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c04a07-22d4-4a00-864b-0faf5278e42b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:10.265526 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c7ebf-488e-412b-a439-5ad43c2b8b87 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:10.265860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c7ebf-488e-412b-a439-5ad43c2b8b87 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:10.266066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c7ebf-488e-412b-a439-5ad43c2b8b87 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:10.267673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 420/835] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:10.272652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9cc72-598a-4b54-a212-8fb865b69f85 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:10.272926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9cc72-598a-4b54-a212-8fb865b69f85 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:10.273231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9cc72-598a-4b54-a212-8fb865b69f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:10.294098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-37a9cc72-598a-4b54-a212-8fb865b69f85 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:10.295256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:10.295596 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-37a9cc72-598a-4b54-a212-8fb865b69f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:10.341697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c04a07-22d4-4a00-864b-0faf5278e42b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:10.381304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9664134-ebfc-43f2-ab50-5c7107454a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:10.382296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9664134-ebfc-43f2-ab50-5c7107454a34 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:10.382770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9664134-ebfc-43f2-ab50-5c7107454a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:10.383251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9664134-ebfc-43f2-ab50-5c7107454a34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:10.385210 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 421/836] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:16:10.394475 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9664134-ebfc-43f2-ab50-5c7107454a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e9664134-ebfc-43f2-ab50-5c7107454a34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:10.399519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:10.400498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:10.401360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:10.401360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:10.401360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:10.401360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:10.401360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:10.401360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:10.401360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:10.401360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:10.401360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:10.401360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:10.401360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:16:10.401360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:10.403853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 416/837] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:16:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:10.409955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9cc72-598a-4b54-a212-8fb865b69f85 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:10.412459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:10.516994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9cc72-598a-4b54-a212-8fb865b69f85 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:10.551038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc97b2c-87ee-4f42-b451-a3b590917552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:10.551643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc97b2c-87ee-4f42-b451-a3b590917552 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:10.551913 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc97b2c-87ee-4f42-b451-a3b590917552 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:10.552194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc97b2c-87ee-4f42-b451-a3b590917552 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:10.552842 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc97b2c-87ee-4f42-b451-a3b590917552 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:10.553159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc97b2c-87ee-4f42-b451-a3b590917552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:10.554584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc97b2c-87ee-4f42-b451-a3b590917552 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:10.563435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 422/838] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:16:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:10.569220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9664134-ebfc-43f2-ab50-5c7107454a34 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:10.671773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9664134-ebfc-43f2-ab50-5c7107454a34 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:10.688037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc97b2c-87ee-4f42-b451-a3b590917552 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:10.701443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:10.712683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea60a1-fa75-4282-a193-681d83cec07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:10.713357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea60a1-fa75-4282-a193-681d83cec07e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:10.713632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea60a1-fa75-4282-a193-681d83cec07e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:10.713898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea60a1-fa75-4282-a193-681d83cec07e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:10.724835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea60a1-fa75-4282-a193-681d83cec07e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:10.726348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea60a1-fa75-4282-a193-681d83cec07e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-13ea60a1-fa75-4282-a193-681d83cec07e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:10.740743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:10.742399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:10.743372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:10.743372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:10.743372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:10.743372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 417/839] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:10.743970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54596ce0-8a8f-4745-ab7f-838a9ad361c4 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:10.744240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54596ce0-8a8f-4745-ab7f-838a9ad361c4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:10.744506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54596ce0-8a8f-4745-ab7f-838a9ad361c4 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:10.749119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 418/840] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:10.749119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54596ce0-8a8f-4745-ab7f-838a9ad361c4 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:10.764984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:10.805543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc97b2c-87ee-4f42-b451-a3b590917552 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:10.840515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a4107-87b4-4c63-84e9-eb0709d7a9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:10.841175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a4107-87b4-4c63-84e9-eb0709d7a9cd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:10.841450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a4107-87b4-4c63-84e9-eb0709d7a9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:10.841911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a4107-87b4-4c63-84e9-eb0709d7a9cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:10.851522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54596ce0-8a8f-4745-ab7f-838a9ad361c4 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:10.851810 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a4107-87b4-4c63-84e9-eb0709d7a9cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f48a4107-87b4-4c63-84e9-eb0709d7a9cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:10.856993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:10.857970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:10.858904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:10.858904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:10.858904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:10.858904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:10.858904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:10.858904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:10.858904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:10.858904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:10.858904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:10.858904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:10.858904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:16:10.858904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:10.863043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 423/841] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:16:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:10.864268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfd470-30fb-4d7d-b889-a13a374cfa5d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:10.864552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfd470-30fb-4d7d-b889-a13a374cfa5d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:10.864813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfd470-30fb-4d7d-b889-a13a374cfa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:10.883509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-03dfd470-30fb-4d7d-b889-a13a374cfa5d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:10.884698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-03dfd470-30fb-4d7d-b889-a13a374cfa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:10.887007 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51a4a4-4eea-41c0-be61-05cf1c1c0749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:10.887699 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51a4a4-4eea-41c0-be61-05cf1c1c0749 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:10.888036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51a4a4-4eea-41c0-be61-05cf1c1c0749 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:10.888373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51a4a4-4eea-41c0-be61-05cf1c1c0749 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:10.889008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51a4a4-4eea-41c0-be61-05cf1c1c0749 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:10.889331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51a4a4-4eea-41c0-be61-05cf1c1c0749 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:10.890667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51a4a4-4eea-41c0-be61-05cf1c1c0749 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:10.906797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 419/842] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:10.906797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51a4a4-4eea-41c0-be61-05cf1c1c0749 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:10.935612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:10.961122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 424/843] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:10.961122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dfd470-30fb-4d7d-b889-a13a374cfa5d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:11.019785 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:11.033671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51a4a4-4eea-41c0-be61-05cf1c1c0749 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:11.073230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279ae0e-18b9-418e-b3fd-956d64ec954d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:11.073913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279ae0e-18b9-418e-b3fd-956d64ec954d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.074296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279ae0e-18b9-418e-b3fd-956d64ec954d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.074617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dfd470-30fb-4d7d-b889-a13a374cfa5d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:11.074732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279ae0e-18b9-418e-b3fd-956d64ec954d None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.075487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279ae0e-18b9-418e-b3fd-956d64ec954d None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:11.075882 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279ae0e-18b9-418e-b3fd-956d64ec954d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:11.078075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279ae0e-18b9-418e-b3fd-956d64ec954d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:11.090827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 420/844] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:16:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:11.091915 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279ae0e-18b9-418e-b3fd-956d64ec954d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:11.111616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cb660-e26a-440a-aeb6-c5e82056cf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:11.112613 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cb660-e26a-440a-aeb6-c5e82056cf60 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.113047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cb660-e26a-440a-aeb6-c5e82056cf60 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.113524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cb660-e26a-440a-aeb6-c5e82056cf60 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.121984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4cb660-e26a-440a-aeb6-c5e82056cf60 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:11.123769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4cb660-e26a-440a-aeb6-c5e82056cf60 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0e4cb660-e26a-440a-aeb6-c5e82056cf60 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:11.137562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:11.138563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:11.139495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:11.139495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:11.139495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:11.139495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:11.139495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:11.139495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:11.139495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:11.139495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:11.139495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 04:16:11.139495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:11.142101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 425/845] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:16:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:11.149135 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cb660-e26a-440a-aeb6-c5e82056cf60 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:11.192142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:11.199460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279ae0e-18b9-418e-b3fd-956d64ec954d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:11.235832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27720d-a779-48d5-bea9-fa9cd16fdabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:11.236475 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27720d-a779-48d5-bea9-fa9cd16fdabd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.236749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27720d-a779-48d5-bea9-fa9cd16fdabd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.237017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27720d-a779-48d5-bea9-fa9cd16fdabd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.247993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27720d-a779-48d5-bea9-fa9cd16fdabd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:11.249453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27720d-a779-48d5-bea9-fa9cd16fdabd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-3d27720d-a779-48d5-bea9-fa9cd16fdabd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:11.263457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:11.264397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:11.265344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:11.265344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:11.265344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:11.269294 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 421/846] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:11.270145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27720d-a779-48d5-bea9-fa9cd16fdabd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:11.279283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cb660-e26a-440a-aeb6-c5e82056cf60 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:11.293742 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:11.315206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b438833-ea99-4988-9e52-52bbf375e611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:11.316096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b438833-ea99-4988-9e52-52bbf375e611 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.316373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b438833-ea99-4988-9e52-52bbf375e611 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.316642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b438833-ea99-4988-9e52-52bbf375e611 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.317602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b438833-ea99-4988-9e52-52bbf375e611 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:11.318275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b438833-ea99-4988-9e52-52bbf375e611 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:11.319710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b438833-ea99-4988-9e52-52bbf375e611 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:11.342774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 426/847] 10.222.0.60 () {56 vars in 1135 bytes} [Sat Sep 17 04:16:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:11.342774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b438833-ea99-4988-9e52-52bbf375e611 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:11.384098 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27720d-a779-48d5-bea9-fa9cd16fdabd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:11.421154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bda6e-cc01-462e-995e-e39de2cb9c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:11.422092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bda6e-cc01-462e-995e-e39de2cb9c45 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.422375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bda6e-cc01-462e-995e-e39de2cb9c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.422797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bda6e-cc01-462e-995e-e39de2cb9c45 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.423387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bda6e-cc01-462e-995e-e39de2cb9c45 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:11.424754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bda6e-cc01-462e-995e-e39de2cb9c45 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-067bda6e-cc01-462e-995e-e39de2cb9c45 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:11.433970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:11.437350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:11.438446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 422/848] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:16:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:11.439453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bda6e-cc01-462e-995e-e39de2cb9c45 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:11.451158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b438833-ea99-4988-9e52-52bbf375e611 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:11.451984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:11.488561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba934c-82cb-4939-aeb0-1801f310c9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:11.489564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba934c-82cb-4939-aeb0-1801f310c9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.489685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba934c-82cb-4939-aeb0-1801f310c9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.489991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba934c-82cb-4939-aeb0-1801f310c9b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.490714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba934c-82cb-4939-aeb0-1801f310c9b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:11.491444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba934c-82cb-4939-aeb0-1801f310c9b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:11.492873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba934c-82cb-4939-aeb0-1801f310c9b5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:11.510944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 427/849] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:11.510944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba934c-82cb-4939-aeb0-1801f310c9b5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:11.541257 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:11.554530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bda6e-cc01-462e-995e-e39de2cb9c45 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:11.590025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7a138-7b87-4023-977f-c88b7c922f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:11.590640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7a138-7b87-4023-977f-c88b7c922f72 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.590915 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7a138-7b87-4023-977f-c88b7c922f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.591182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7a138-7b87-4023-977f-c88b7c922f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.615925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba934c-82cb-4939-aeb0-1801f310c9b5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:11.650102 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551c97a-f94c-4f52-8a6e-851b667cc6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:11.650701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551c97a-f94c-4f52-8a6e-851b667cc6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.650970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551c97a-f94c-4f52-8a6e-851b667cc6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.651231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551c97a-f94c-4f52-8a6e-851b667cc6a3 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.651851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551c97a-f94c-4f52-8a6e-851b667cc6a3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:11.652143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551c97a-f94c-4f52-8a6e-851b667cc6a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:11.653533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551c97a-f94c-4f52-8a6e-851b667cc6a3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:11.657705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 428/850] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:16:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:11.658471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551c97a-f94c-4f52-8a6e-851b667cc6a3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:11.662579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7a138-7b87-4023-977f-c88b7c922f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:11.664118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7a138-7b87-4023-977f-c88b7c922f72 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:11.676486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:11.757054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:11.762991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551c97a-f94c-4f52-8a6e-851b667cc6a3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:11.763645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 423/851] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:16:11 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:16:11.764871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d19d04-983e-4715-a7f9-066c2551be31 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.765206 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d19d04-983e-4715-a7f9-066c2551be31 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.765458 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d19d04-983e-4715-a7f9-066c2551be31 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.766666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 424/852] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:11.767352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d19d04-983e-4715-a7f9-066c2551be31 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:11.800992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a4ada-6316-4761-aaa0-867b6057c590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:11.801656 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a4ada-6316-4761-aaa0-867b6057c590 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.801934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a4ada-6316-4761-aaa0-867b6057c590 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.802200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a4ada-6316-4761-aaa0-867b6057c590 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.809496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a4ada-6316-4761-aaa0-867b6057c590 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:11.810980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a4ada-6316-4761-aaa0-867b6057c590 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-076a4ada-6316-4761-aaa0-867b6057c590 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:11.821961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:11.823082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:11.824216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:11.824216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:11.824216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:11.824216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:11.824216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:11.824216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:11.824216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:11.824216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:11.824216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 04:16:11.824216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:11.824216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 429/853] 10.222.0.60 () {56 vars in 1117 bytes} [Sat Sep 17 04:16:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:11.824216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a4ada-6316-4761-aaa0-867b6057c590 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:11.875088 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d19d04-983e-4715-a7f9-066c2551be31 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:11.886540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:11.910293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7271bb-ff7e-4245-9e24-a8100741c5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:11.911035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7271bb-ff7e-4245-9e24-a8100741c5ea None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.911237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7271bb-ff7e-4245-9e24-a8100741c5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.911432 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7271bb-ff7e-4245-9e24-a8100741c5ea None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.912065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7271bb-ff7e-4245-9e24-a8100741c5ea None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:11.912361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7271bb-ff7e-4245-9e24-a8100741c5ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:11.913756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7271bb-ff7e-4245-9e24-a8100741c5ea None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:11.919409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 425/854] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:16:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:11.920556 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395c01f-69ee-4991-b105-b31b4d5a21b6 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.920859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395c01f-69ee-4991-b105-b31b4d5a21b6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.921128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395c01f-69ee-4991-b105-b31b4d5a21b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.923702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a4ada-6316-4761-aaa0-867b6057c590 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:11.940727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6395c01f-69ee-4991-b105-b31b4d5a21b6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:11.942001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6395c01f-69ee-4991-b105-b31b4d5a21b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:11.965522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1ab33-d802-4fc2-b18b-9701b3c7eb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:11.966261 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1ab33-d802-4fc2-b18b-9701b3c7eb75 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:11.966592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1ab33-d802-4fc2-b18b-9701b3c7eb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:11.966918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1ab33-d802-4fc2-b18b-9701b3c7eb75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:11.979020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1ab33-d802-4fc2-b18b-9701b3c7eb75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:11.980626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1ab33-d802-4fc2-b18b-9701b3c7eb75 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c6b1ab33-d802-4fc2-b18b-9701b3c7eb75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:11.997964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:11.999238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:12.000377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:12.000377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:12.000377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:12.000931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 430/855] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:12.001534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1ab33-d802-4fc2-b18b-9701b3c7eb75 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:12.014282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:12.024120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 426/856] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:12.025370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395c01f-69ee-4991-b105-b31b4d5a21b6 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:12.139905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1ab33-d802-4fc2-b18b-9701b3c7eb75 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:12.153572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395c01f-69ee-4991-b105-b31b4d5a21b6 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:12.156803 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:12.188240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb9712-253d-435e-b277-5c95a4f201fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:12.188846 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb9712-253d-435e-b277-5c95a4f201fb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:12.189149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb9712-253d-435e-b277-5c95a4f201fb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:12.189420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb9712-253d-435e-b277-5c95a4f201fb None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:12.190153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb9712-253d-435e-b277-5c95a4f201fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:12.190463 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb9712-253d-435e-b277-5c95a4f201fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:12.191867 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb9712-253d-435e-b277-5c95a4f201fb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:12.193166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952ae5b-aaf9-4eeb-a04a-14a4b378878b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:12.194146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952ae5b-aaf9-4eeb-a04a-14a4b378878b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:12.195320 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952ae5b-aaf9-4eeb-a04a-14a4b378878b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:12.195744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952ae5b-aaf9-4eeb-a04a-14a4b378878b None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:12.196571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952ae5b-aaf9-4eeb-a04a-14a4b378878b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:12.197031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952ae5b-aaf9-4eeb-a04a-14a4b378878b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:12.198574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952ae5b-aaf9-4eeb-a04a-14a4b378878b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:12.213066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 427/857] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:12.217601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 431/858] 10.222.0.60 () {56 vars in 1149 bytes} [Sat Sep 17 04:16:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:12.217601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb9712-253d-435e-b277-5c95a4f201fb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:12.218373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952ae5b-aaf9-4eeb-a04a-14a4b378878b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:12.305960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:12.326271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952ae5b-aaf9-4eeb-a04a-14a4b378878b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:12.346119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb9712-253d-435e-b277-5c95a4f201fb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:12.363431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da111882-2384-4575-b15c-cbd31eac2328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:12.364565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da111882-2384-4575-b15c-cbd31eac2328 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:12.365041 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da111882-2384-4575-b15c-cbd31eac2328 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:12.365511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da111882-2384-4575-b15c-cbd31eac2328 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:12.373193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da111882-2384-4575-b15c-cbd31eac2328 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:12.374617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da111882-2384-4575-b15c-cbd31eac2328 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:12.380866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22783bbd-4ebc-41ec-82a5-41da07929df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:12.381508 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22783bbd-4ebc-41ec-82a5-41da07929df6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:12.381781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22783bbd-4ebc-41ec-82a5-41da07929df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:12.382052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22783bbd-4ebc-41ec-82a5-41da07929df6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-da111882-2384-4575-b15c-cbd31eac2328 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:12.386184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:12.387045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:12.388420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:12.388420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:12.388420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:12.388420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:12.388420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:12.388420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:12.388420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:12.388420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:12.388420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 428/859] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:12.388420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da111882-2384-4575-b15c-cbd31eac2328 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:12.416430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:12.458389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22783bbd-4ebc-41ec-82a5-41da07929df6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:12.459836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22783bbd-4ebc-41ec-82a5-41da07929df6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:12.494124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da111882-2384-4575-b15c-cbd31eac2328 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:12.521455 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:12.530514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105c256-73f3-4df6-a956-5c1f1e7037b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:12.531257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105c256-73f3-4df6-a956-5c1f1e7037b5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:12.531538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105c256-73f3-4df6-a956-5c1f1e7037b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:12.531805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105c256-73f3-4df6-a956-5c1f1e7037b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:12.532377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105c256-73f3-4df6-a956-5c1f1e7037b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:12.533781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105c256-73f3-4df6-a956-5c1f1e7037b5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-3105c256-73f3-4df6-a956-5c1f1e7037b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:12.542912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:12.543893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:12.546158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 429/860] 10.222.0.60 () {56 vars in 1132 bytes} [Sat Sep 17 04:16:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:12.546960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105c256-73f3-4df6-a956-5c1f1e7037b5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:12.566670 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 432/861] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:16:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:16:12.567522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22783bbd-4ebc-41ec-82a5-41da07929df6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:12.642527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:12.655562 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105c256-73f3-4df6-a956-5c1f1e7037b5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:12.678034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22783bbd-4ebc-41ec-82a5-41da07929df6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:12.693268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b74f8e-fab2-42d0-9977-1d34977fcb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:12.693886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b74f8e-fab2-42d0-9977-1d34977fcb1c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:12.694165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b74f8e-fab2-42d0-9977-1d34977fcb1c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:12.694445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b74f8e-fab2-42d0-9977-1d34977fcb1c None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:12.695129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b74f8e-fab2-42d0-9977-1d34977fcb1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:12.695430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b74f8e-fab2-42d0-9977-1d34977fcb1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:12.696931 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b74f8e-fab2-42d0-9977-1d34977fcb1c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:12.714048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 430/862] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:12.714048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b74f8e-fab2-42d0-9977-1d34977fcb1c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:12.719370 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179fa49-c425-4275-9019-fe5f42a19d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:12.720064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179fa49-c425-4275-9019-fe5f42a19d0c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:12.720538 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179fa49-c425-4275-9019-fe5f42a19d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:12.720745 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179fa49-c425-4275-9019-fe5f42a19d0c None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:12.721478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179fa49-c425-4275-9019-fe5f42a19d0c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:12.721848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179fa49-c425-4275-9019-fe5f42a19d0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:12.723292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179fa49-c425-4275-9019-fe5f42a19d0c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:12.728047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 433/863] 10.222.0.60 () {56 vars in 1126 bytes} [Sat Sep 17 04:16:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:12.728738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179fa49-c425-4275-9019-fe5f42a19d0c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:12.759971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:12.833677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b74f8e-fab2-42d0-9977-1d34977fcb1c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:12.843625 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179fa49-c425-4275-9019-fe5f42a19d0c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:12.867759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ea64d-fb27-4240-ba44-6a90870fcb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:12.868487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ea64d-fb27-4240-ba44-6a90870fcb29 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:12.868764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ea64d-fb27-4240-ba44-6a90870fcb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:12.869036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ea64d-fb27-4240-ba44-6a90870fcb29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:12.878600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33997031-4df5-4466-be08-dd18a49df233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:12.878907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ea64d-fb27-4240-ba44-6a90870fcb29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:12.879208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33997031-4df5-4466-be08-dd18a49df233 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:12.879484 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33997031-4df5-4466-be08-dd18a49df233 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:12.879750 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33997031-4df5-4466-be08-dd18a49df233 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:12.880332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ea64d-fb27-4240-ba44-6a90870fcb29 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:12.880495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33997031-4df5-4466-be08-dd18a49df233 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=7aac7cd1742449328facf2902483f259) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:12.880762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33997031-4df5-4466-be08-dd18a49df233 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:12.882205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33997031-4df5-4466-be08-dd18a49df233 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:12.897111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 434/864] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:16:12 2022] GET /identity/v3/role_assignments?user.id=7aac7cd1742449328facf2902483f259&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:12.897111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33997031-4df5-4466-be08-dd18a49df233 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:12.898776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c66ea64d-fb27-4240-ba44-6a90870fcb29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:12.908423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:12.909581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:12.911006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:12.911006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:12.911006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:12.913363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 431/865] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:12.916347 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ea64d-fb27-4240-ba44-6a90870fcb29 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:13.008717 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:13.017197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33997031-4df5-4466-be08-dd18a49df233 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:13.033913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ea64d-fb27-4240-ba44-6a90870fcb29 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:13.055449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1ef96-647e-4e03-bfbf-ff190ef57d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:13.056292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1ef96-647e-4e03-bfbf-ff190ef57d24 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:13.056689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1ef96-647e-4e03-bfbf-ff190ef57d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:13.057047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1ef96-647e-4e03-bfbf-ff190ef57d24 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:13.057794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1ef96-647e-4e03-bfbf-ff190ef57d24 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:13.058163 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1ef96-647e-4e03-bfbf-ff190ef57d24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:13.059661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1ef96-647e-4e03-bfbf-ff190ef57d24 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:13.065524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 435/866] 10.222.0.60 () {56 vars in 1126 bytes} [Sat Sep 17 04:16:12 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:13.077702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ce7ae-57d6-4c06-9d1d-ef20c6c49607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:13.078564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ce7ae-57d6-4c06-9d1d-ef20c6c49607 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:13.078936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ce7ae-57d6-4c06-9d1d-ef20c6c49607 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:13.079354 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ce7ae-57d6-4c06-9d1d-ef20c6c49607 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:13.081210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ce7ae-57d6-4c06-9d1d-ef20c6c49607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:13.081210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ce7ae-57d6-4c06-9d1d-ef20c6c49607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:13.081952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ce7ae-57d6-4c06-9d1d-ef20c6c49607 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:13.095769 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 432/867] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:13.102338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1ef96-647e-4e03-bfbf-ff190ef57d24 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:13.218323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1ef96-647e-4e03-bfbf-ff190ef57d24 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:13.256230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41376b8e-2d95-49df-b6ef-cd0970a394dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:13.256869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41376b8e-2d95-49df-b6ef-cd0970a394dc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:13.257174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41376b8e-2d95-49df-b6ef-cd0970a394dc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:13.257448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41376b8e-2d95-49df-b6ef-cd0970a394dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:13.264584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41376b8e-2d95-49df-b6ef-cd0970a394dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:13.266070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41376b8e-2d95-49df-b6ef-cd0970a394dc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-41376b8e-2d95-49df-b6ef-cd0970a394dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:13.277698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:13.278610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:13.279876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:13.279876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:13.279876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:13.279876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:13.279876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:13.279876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:13.279876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:13.279876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:13.279876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 436/868] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:13.284900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ce7ae-57d6-4c06-9d1d-ef20c6c49607 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:13.347060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:13.390145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ce7ae-57d6-4c06-9d1d-ef20c6c49607 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:13.423983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84657e-5f52-485d-be83-8dcce0f870b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:13.424589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84657e-5f52-485d-be83-8dcce0f870b0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:13.424955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84657e-5f52-485d-be83-8dcce0f870b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:13.425253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84657e-5f52-485d-be83-8dcce0f870b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:13.425937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84657e-5f52-485d-be83-8dcce0f870b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:13.426238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84657e-5f52-485d-be83-8dcce0f870b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:13.427719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84657e-5f52-485d-be83-8dcce0f870b0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:13.439916 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 433/869] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:13.446192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41376b8e-2d95-49df-b6ef-cd0970a394dc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:13.466578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:13.549661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41376b8e-2d95-49df-b6ef-cd0970a394dc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:13.583983 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44b658-a739-47ee-8510-e9db44605e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:13.584994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44b658-a739-47ee-8510-e9db44605e41 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:13.585535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44b658-a739-47ee-8510-e9db44605e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:13.586007 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44b658-a739-47ee-8510-e9db44605e41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:13.587046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44b658-a739-47ee-8510-e9db44605e41 None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=7c1a7d2677184c24ade81c397b7d09d1) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:13.587565 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44b658-a739-47ee-8510-e9db44605e41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:13.589240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44b658-a739-47ee-8510-e9db44605e41 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:13.595699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 437/870] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:16:13 2022] GET /identity/v3/role_assignments?user.id=7c1a7d2677184c24ade81c397b7d09d1&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:14.085591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2ac9f-93c3-4762-944f-f6d358604a70 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:14.086176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2ac9f-93c3-4762-944f-f6d358604a70 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:14.086176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2ac9f-93c3-4762-944f-f6d358604a70 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:14.087794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 434/871] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:14.092940 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb871abf-c019-4930-b0ff-b80555b5b093 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:14.093253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb871abf-c019-4930-b0ff-b80555b5b093 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:14.093527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb871abf-c019-4930-b0ff-b80555b5b093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:14.115423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-eb871abf-c019-4930-b0ff-b80555b5b093 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:14.116977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eb871abf-c019-4930-b0ff-b80555b5b093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:14.183260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 438/872] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:14.445665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2ac9f-93c3-4762-944f-f6d358604a70 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:14.547976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2ac9f-93c3-4762-944f-f6d358604a70 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:14.568337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:14.583470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4570d-f1cf-4e71-9133-4a567521c47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:14.584086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4570d-f1cf-4e71-9133-4a567521c47b None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:14.584364 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4570d-f1cf-4e71-9133-4a567521c47b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:14.584631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4570d-f1cf-4e71-9133-4a567521c47b None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:14.585318 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4570d-f1cf-4e71-9133-4a567521c47b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:14.586677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4570d-f1cf-4e71-9133-4a567521c47b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:14.608050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 435/873] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:16:14 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:16:15.157244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f78429-2baf-4e6c-8a05-abcb7bdd5f39 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:15.157785 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f78429-2baf-4e6c-8a05-abcb7bdd5f39 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:15.157785 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f78429-2baf-4e6c-8a05-abcb7bdd5f39 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:15.159385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 439/874] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:15.164628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffe895-7119-492e-94d3-627896f1b2b6 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:15.165067 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffe895-7119-492e-94d3-627896f1b2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:15.165264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffe895-7119-492e-94d3-627896f1b2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:15.184015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0dffe895-7119-492e-94d3-627896f1b2b6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:15.185267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0dffe895-7119-492e-94d3-627896f1b2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:15.265398 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 436/875] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:15.333399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f5403-7d96-4788-89f6-8d8519ed92de None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:15.333719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f5403-7d96-4788-89f6-8d8519ed92de None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:15.333943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f5403-7d96-4788-89f6-8d8519ed92de None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:15.335550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 440/876] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:15.340864 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b7c1a-ae38-44d9-b487-24b579890333 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:15.341176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b7c1a-ae38-44d9-b487-24b579890333 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:15.341449 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b7c1a-ae38-44d9-b487-24b579890333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:15.361119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2a7b7c1a-ae38-44d9-b487-24b579890333 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:15.362401 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2a7b7c1a-ae38-44d9-b487-24b579890333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:15.430723 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 437/877] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:15.537637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f5403-7d96-4788-89f6-8d8519ed92de None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:15.552889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:15.644301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f5403-7d96-4788-89f6-8d8519ed92de None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:15.689001 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b0227-a70d-46ef-876d-6b8b79a327c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:15.689607 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b0227-a70d-46ef-876d-6b8b79a327c6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:15.690039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b0227-a70d-46ef-876d-6b8b79a327c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:15.690310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b0227-a70d-46ef-876d-6b8b79a327c6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:15.690892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b0227-a70d-46ef-876d-6b8b79a327c6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:15.692255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b0227-a70d-46ef-876d-6b8b79a327c6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-757b0227-a70d-46ef-876d-6b8b79a327c6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:15.701239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:15.703139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:15.704175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 441/878] 10.222.0.60 () {56 vars in 1126 bytes} [Sat Sep 17 04:16:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:15.708416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b7c1a-ae38-44d9-b487-24b579890333 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:15.717858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b0227-a70d-46ef-876d-6b8b79a327c6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:15.751373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:15.808187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:15.834590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b0227-a70d-46ef-876d-6b8b79a327c6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:15.840267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b7c1a-ae38-44d9-b487-24b579890333 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:15.874329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4b5b8-199d-4d08-b232-e2618212988b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:15.875126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4b5b8-199d-4d08-b232-e2618212988b None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:15.875510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4b5b8-199d-4d08-b232-e2618212988b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:15.875843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4b5b8-199d-4d08-b232-e2618212988b None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:15.876559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4b5b8-199d-4d08-b232-e2618212988b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:15.878010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4b5b8-199d-4d08-b232-e2618212988b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:15.881785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddba9f-7a56-4b1b-92f9-4424b38a55d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:15.883048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddba9f-7a56-4b1b-92f9-4424b38a55d2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:15.883532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddba9f-7a56-4b1b-92f9-4424b38a55d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:15.884205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddba9f-7a56-4b1b-92f9-4424b38a55d2 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:15.885023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ddba9f-7a56-4b1b-92f9-4424b38a55d2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:15.885454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ddba9f-7a56-4b1b-92f9-4424b38a55d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:15.887036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ddba9f-7a56-4b1b-92f9-4424b38a55d2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:15.896446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 442/879] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:16:15 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:16:15.898749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 438/880] 10.222.0.60 () {56 vars in 1122 bytes} [Sat Sep 17 04:16:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:15.898979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e597dde-b5c6-454d-919a-95ad2c05e4f9 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:15.899139 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddba9f-7a56-4b1b-92f9-4424b38a55d2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:15.899290 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e597dde-b5c6-454d-919a-95ad2c05e4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:15.899290 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e597dde-b5c6-454d-919a-95ad2c05e4f9 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:15.900585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 443/881] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:15.908671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651591b-e392-4e4a-b558-9ccb5fb8413b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:15.909000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651591b-e392-4e4a-b558-9ccb5fb8413b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:15.909308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651591b-e392-4e4a-b558-9ccb5fb8413b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:15.936284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c651591b-e392-4e4a-b558-9ccb5fb8413b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:15.937583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c651591b-e392-4e4a-b558-9ccb5fb8413b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:16.028843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:16.035307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddba9f-7a56-4b1b-92f9-4424b38a55d2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:16.036299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 444/882] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:16.079005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99e8ec-7e0b-49c7-bf9c-de6c251298a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:16.080053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99e8ec-7e0b-49c7-bf9c-de6c251298a9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:16.080528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99e8ec-7e0b-49c7-bf9c-de6c251298a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:16.081000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99e8ec-7e0b-49c7-bf9c-de6c251298a9 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:16.081866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99e8ec-7e0b-49c7-bf9c-de6c251298a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:16.082363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99e8ec-7e0b-49c7-bf9c-de6c251298a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:16.083947 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99e8ec-7e0b-49c7-bf9c-de6c251298a9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:16.089099 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 439/883] 10.222.0.60 () {56 vars in 1122 bytes} [Sat Sep 17 04:16:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:16.318251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651591b-e392-4e4a-b558-9ccb5fb8413b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:16.381991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:16.422906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651591b-e392-4e4a-b558-9ccb5fb8413b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:16.458593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b37268-05a8-4185-b1ad-4d1e40dee3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:16.459492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b37268-05a8-4185-b1ad-4d1e40dee3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:16.459898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b37268-05a8-4185-b1ad-4d1e40dee3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:16.460322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b37268-05a8-4185-b1ad-4d1e40dee3f2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:16.472535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b37268-05a8-4185-b1ad-4d1e40dee3f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-37b37268-05a8-4185-b1ad-4d1e40dee3f2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:16.477856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:16.478806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:16.479668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 445/884] 10.222.0.60 () {56 vars in 1120 bytes} [Sat Sep 17 04:16:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:16.486734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99e8ec-7e0b-49c7-bf9c-de6c251298a9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:16.590273 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99e8ec-7e0b-49c7-bf9c-de6c251298a9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:16.624944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6824470-0068-46e0-aa9d-bcee337ec805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:16.625593 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6824470-0068-46e0-aa9d-bcee337ec805 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:16.625869 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6824470-0068-46e0-aa9d-bcee337ec805 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:16.626140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6824470-0068-46e0-aa9d-bcee337ec805 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:16.626809 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6824470-0068-46e0-aa9d-bcee337ec805 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:16.627107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6824470-0068-46e0-aa9d-bcee337ec805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:16.628473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6824470-0068-46e0-aa9d-bcee337ec805 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:16.637331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 440/885] 10.222.0.60 () {56 vars in 1115 bytes} [Sat Sep 17 04:16:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:16.643123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b37268-05a8-4185-b1ad-4d1e40dee3f2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:16.748021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b37268-05a8-4185-b1ad-4d1e40dee3f2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:16.771488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:16.785787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a6375-427d-4acd-8dde-69df64b023f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:16.787033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a6375-427d-4acd-8dde-69df64b023f8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:16.787698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a6375-427d-4acd-8dde-69df64b023f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:16.787970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a6375-427d-4acd-8dde-69df64b023f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:16.799922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a6375-427d-4acd-8dde-69df64b023f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:16.801728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a6375-427d-4acd-8dde-69df64b023f8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-2c1a6375-427d-4acd-8dde-69df64b023f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:16.818584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:16.819734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:16.820609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:16.820609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:16.820609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:16.820609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 446/886] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:16.826289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6824470-0068-46e0-aa9d-bcee337ec805 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:16.857573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:16.914803 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d7006-b213-4eec-b20e-a2a019f3b7ba None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:16.915236 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d7006-b213-4eec-b20e-a2a019f3b7ba None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:16.915425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d7006-b213-4eec-b20e-a2a019f3b7ba None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:16.916921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 447/887] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:16:16.922774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda50edc-8f85-494d-a57d-143b907a57a1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:16.922774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda50edc-8f85-494d-a57d-143b907a57a1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:16.922774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda50edc-8f85-494d-a57d-143b907a57a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:16.937344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6824470-0068-46e0-aa9d-bcee337ec805 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:16.941657 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-eda50edc-8f85-494d-a57d-143b907a57a1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:16.943068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eda50edc-8f85-494d-a57d-143b907a57a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:16.978272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846df17-2971-49fb-863f-83a338243529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:16.979057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846df17-2971-49fb-863f-83a338243529 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:16.979339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846df17-2971-49fb-863f-83a338243529 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:16.979614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846df17-2971-49fb-863f-83a338243529 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:16.981112 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846df17-2971-49fb-863f-83a338243529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:16.981440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846df17-2971-49fb-863f-83a338243529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:16.982852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846df17-2971-49fb-863f-83a338243529 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:17.000263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 441/888] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:17.006090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846df17-2971-49fb-863f-83a338243529 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:17.025242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 448/889] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:16:17.094484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:17.110110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846df17-2971-49fb-863f-83a338243529 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:17.144849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d545e0d-e605-474b-a8d0-59bda1d1f7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:17.145838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d545e0d-e605-474b-a8d0-59bda1d1f7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:17.146331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d545e0d-e605-474b-a8d0-59bda1d1f7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:17.146798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d545e0d-e605-474b-a8d0-59bda1d1f7a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:17.154090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d545e0d-e605-474b-a8d0-59bda1d1f7a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:17.155745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d545e0d-e605-474b-a8d0-59bda1d1f7a3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-7d545e0d-e605-474b-a8d0-59bda1d1f7a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:17.166934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:17.167902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:17.168867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:17.168867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:17.168867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:17.168867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:17.168867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:17.168867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:17.168867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:17.168867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:17.168867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 04:16:17.168867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:17.171368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 442/890] 10.222.0.60 () {56 vars in 1117 bytes} [Sat Sep 17 04:16:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:17.177345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda50edc-8f85-494d-a57d-143b907a57a1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:17.276101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda50edc-8f85-494d-a57d-143b907a57a1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:17.278888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:17.288605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d545e0d-e605-474b-a8d0-59bda1d1f7a3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:17.316327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd9a7c-3985-47c9-9792-c5653373d557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:17.317002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd9a7c-3985-47c9-9792-c5653373d557 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:17.317482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd9a7c-3985-47c9-9792-c5653373d557 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:17.317843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd9a7c-3985-47c9-9792-c5653373d557 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:17.318974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd9a7c-3985-47c9-9792-c5653373d557 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:17.319599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd9a7c-3985-47c9-9792-c5653373d557 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:17.321490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd9a7c-3985-47c9-9792-c5653373d557 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:17.353239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 449/891] 10.222.0.60 () {56 vars in 1149 bytes} [Sat Sep 17 04:16:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:17.355010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd9a7c-3985-47c9-9792-c5653373d557 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:17.397343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d545e0d-e605-474b-a8d0-59bda1d1f7a3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:17.439596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5612fb8-f590-4003-a567-ba6a5989128d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:17.440384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5612fb8-f590-4003-a567-ba6a5989128d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:17.440756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5612fb8-f590-4003-a567-ba6a5989128d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:17.441168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5612fb8-f590-4003-a567-ba6a5989128d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:17.441892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5612fb8-f590-4003-a567-ba6a5989128d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:17.443768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5612fb8-f590-4003-a567-ba6a5989128d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d5612fb8-f590-4003-a567-ba6a5989128d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:17.454788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:17.455976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:17.456908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 443/892] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:16:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:17.462701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5612fb8-f590-4003-a567-ba6a5989128d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:17.469966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd9a7c-3985-47c9-9792-c5653373d557 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:17.497460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:17.507336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea3f4e-bf5b-4500-ba19-3697a9d64516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:17.507947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea3f4e-bf5b-4500-ba19-3697a9d64516 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:17.508227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea3f4e-bf5b-4500-ba19-3697a9d64516 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:17.508592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea3f4e-bf5b-4500-ba19-3697a9d64516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:17.541262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:17.588730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5612fb8-f590-4003-a567-ba6a5989128d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:17.593437 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea3f4e-bf5b-4500-ba19-3697a9d64516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:17.595111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea3f4e-bf5b-4500-ba19-3697a9d64516 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:17.624203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859bc76-d9ac-4ac4-9d19-1bde2d853824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:17.624600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859bc76-d9ac-4ac4-9d19-1bde2d853824 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:17.624880 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859bc76-d9ac-4ac4-9d19-1bde2d853824 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:17.625177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859bc76-d9ac-4ac4-9d19-1bde2d853824 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:17.625813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859bc76-d9ac-4ac4-9d19-1bde2d853824 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:17.626114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859bc76-d9ac-4ac4-9d19-1bde2d853824 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:17.627476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859bc76-d9ac-4ac4-9d19-1bde2d853824 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:17.637200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 444/893] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:16:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:17.638996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859bc76-d9ac-4ac4-9d19-1bde2d853824 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:17.712713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 450/894] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:16:17 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 04:16:17.721737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea3f4e-bf5b-4500-ba19-3697a9d64516 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:17.732020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:17.758193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859bc76-d9ac-4ac4-9d19-1bde2d853824 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:17.794719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab2fbe-ef6d-4d52-a3b3-3301ea868bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:17.795593 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab2fbe-ef6d-4d52-a3b3-3301ea868bcb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:17.796009 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab2fbe-ef6d-4d52-a3b3-3301ea868bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:17.796439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab2fbe-ef6d-4d52-a3b3-3301ea868bcb None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:17.797369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab2fbe-ef6d-4d52-a3b3-3301ea868bcb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:17.797956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab2fbe-ef6d-4d52-a3b3-3301ea868bcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:17.799747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:17.800057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab2fbe-ef6d-4d52-a3b3-3301ea868bcb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:17.807607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 445/895] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:16:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:17.812414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab2fbe-ef6d-4d52-a3b3-3301ea868bcb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:17.852923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea3f4e-bf5b-4500-ba19-3697a9d64516 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:17.897464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9335cd1-6c52-446d-a492-91676810f500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:17.898523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9335cd1-6c52-446d-a492-91676810f500 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:17.898975 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9335cd1-6c52-446d-a492-91676810f500 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:17.899605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9335cd1-6c52-446d-a492-91676810f500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:17.913096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9335cd1-6c52-446d-a492-91676810f500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:17.915558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9335cd1-6c52-446d-a492-91676810f500 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:17.933988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab2fbe-ef6d-4d52-a3b3-3301ea868bcb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f9335cd1-6c52-446d-a492-91676810f500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:17.934418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:17.935414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:17.936303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:17.936303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:17.936303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:17.938996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 451/896] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:17.945123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9335cd1-6c52-446d-a492-91676810f500 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:17.983111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45de200-e07f-4b3f-8f18-ea68ce5a4cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:17.984097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45de200-e07f-4b3f-8f18-ea68ce5a4cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:17.984605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45de200-e07f-4b3f-8f18-ea68ce5a4cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:17.985070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45de200-e07f-4b3f-8f18-ea68ce5a4cc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:17.986522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45de200-e07f-4b3f-8f18-ea68ce5a4cc1 None admin] RBAC: Adding query filter params (interface=public, service_id=c8ae26d6ae53472280b33a85e3e0911d, region_id=RegionOne) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:17.987071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45de200-e07f-4b3f-8f18-ea68ce5a4cc1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:17.989412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45de200-e07f-4b3f-8f18-ea68ce5a4cc1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:17.997504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 446/897] 10.222.0.60 () {56 vars in 1251 bytes} [Sat Sep 17 04:16:17 2022] GET /identity/v3/endpoints?service_id=c8ae26d6ae53472280b33a85e3e0911d&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:18.005066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45de200-e07f-4b3f-8f18-ea68ce5a4cc1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:18.009175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:18.069249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9335cd1-6c52-446d-a492-91676810f500 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:18.091076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:18.122668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa8dbe-5ec9-4f32-9441-cef0b9813662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:18.123866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa8dbe-5ec9-4f32-9441-cef0b9813662 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:18.124571 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa8dbe-5ec9-4f32-9441-cef0b9813662 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:18.125023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa8dbe-5ec9-4f32-9441-cef0b9813662 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:18.126477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa8dbe-5ec9-4f32-9441-cef0b9813662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:18.127004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa8dbe-5ec9-4f32-9441-cef0b9813662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:18.129620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa8dbe-5ec9-4f32-9441-cef0b9813662 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:18.134487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45de200-e07f-4b3f-8f18-ea68ce5a4cc1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:18.150218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 452/898] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:18.157407 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa8dbe-5ec9-4f32-9441-cef0b9813662 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:18.180443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9845c-5836-46d8-b82a-235dc5e5e41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:18.181199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9845c-5836-46d8-b82a-235dc5e5e41c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:18.181485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9845c-5836-46d8-b82a-235dc5e5e41c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:18.181813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9845c-5836-46d8-b82a-235dc5e5e41c None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:18.182435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9845c-5836-46d8-b82a-235dc5e5e41c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:18.183827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9845c-5836-46d8-b82a-235dc5e5e41c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:18.190936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 447/899] 10.222.0.60 () {56 vars in 1087 bytes} [Sat Sep 17 04:16:18 2022] GET /identity/v3/services => generated 1268 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:16:18.192090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6549fe-3203-464e-a234-a5b04b328a6c None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:18.192376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6549fe-3203-464e-a234-a5b04b328a6c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:18.192647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6549fe-3203-464e-a234-a5b04b328a6c None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:18.197982 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 448/900] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:18.199737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012afb6-283a-45b4-8489-03d9fd1a59a1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:18.200017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012afb6-283a-45b4-8489-03d9fd1a59a1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:18.200289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012afb6-283a-45b4-8489-03d9fd1a59a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:18.226218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e012afb6-283a-45b4-8489-03d9fd1a59a1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:18.227783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e012afb6-283a-45b4-8489-03d9fd1a59a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:18.294090 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa8dbe-5ec9-4f32-9441-cef0b9813662 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:18.306061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:18.314934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 449/901] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:18.322659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd265aed-31b5-4e70-b803-d2b2474ab1a3 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:18.323814 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd265aed-31b5-4e70-b803-d2b2474ab1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:18.324227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd265aed-31b5-4e70-b803-d2b2474ab1a3 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:18.330497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 450/902] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:18 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:18.333762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d224e-7790-4110-9bc1-75093f95f37d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:18.334296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d224e-7790-4110-9bc1-75093f95f37d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:18.335040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d224e-7790-4110-9bc1-75093f95f37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:18.343710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc80ca8-fb90-4272-97b7-d059ed1fde90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:18.344421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc80ca8-fb90-4272-97b7-d059ed1fde90 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:18.344692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc80ca8-fb90-4272-97b7-d059ed1fde90 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:18.344961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc80ca8-fb90-4272-97b7-d059ed1fde90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:18.354934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc80ca8-fb90-4272-97b7-d059ed1fde90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:18.356709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc80ca8-fb90-4272-97b7-d059ed1fde90 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:18.361274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f09d224e-7790-4110-9bc1-75093f95f37d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:18.362850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f09d224e-7790-4110-9bc1-75093f95f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-7bc80ca8-fb90-4272-97b7-d059ed1fde90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:18.371262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:18.372219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:18.373110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:18.373110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:18.373110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:18.373110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:18.373110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:18.373110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:18.373110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:18.373110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:18.377241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 453/903] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:18.380825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc80ca8-fb90-4272-97b7-d059ed1fde90 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:18.429853 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:18.454518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 451/904] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:18.512039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc80ca8-fb90-4272-97b7-d059ed1fde90 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:18.553669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980317b3-eb8e-48cc-ab0d-6765a1e25a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:18.554555 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980317b3-eb8e-48cc-ab0d-6765a1e25a25 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:18.554984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980317b3-eb8e-48cc-ab0d-6765a1e25a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:18.555445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980317b3-eb8e-48cc-ab0d-6765a1e25a25 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:18.556710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980317b3-eb8e-48cc-ab0d-6765a1e25a25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:18.557266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980317b3-eb8e-48cc-ab0d-6765a1e25a25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:18.559860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980317b3-eb8e-48cc-ab0d-6765a1e25a25 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:18.574842 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 454/905] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:18.581197 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d224e-7790-4110-9bc1-75093f95f37d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:18.582072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980317b3-eb8e-48cc-ab0d-6765a1e25a25 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:18.701855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:18.710594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d224e-7790-4110-9bc1-75093f95f37d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:18.713673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980317b3-eb8e-48cc-ab0d-6765a1e25a25 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:18.741783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:18.750046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aa6a3-26e7-4229-9ed8-afbf4d85c4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:18.750672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aa6a3-26e7-4229-9ed8-afbf4d85c4ac None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:18.750932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753046c-5b70-4f95-8c1a-86a43216af87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:18.751066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aa6a3-26e7-4229-9ed8-afbf4d85c4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:18.751239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aa6a3-26e7-4229-9ed8-afbf4d85c4ac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:18.751535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753046c-5b70-4f95-8c1a-86a43216af87 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:18.751788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8aa6a3-26e7-4229-9ed8-afbf4d85c4ac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:18.751892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753046c-5b70-4f95-8c1a-86a43216af87 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:18.752053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753046c-5b70-4f95-8c1a-86a43216af87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:18.752770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753046c-5b70-4f95-8c1a-86a43216af87 None admin] RBAC: Adding query filter params (role.id=ed3c392a05fc45baac04fa3092869141, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=7c1a7d2677184c24ade81c397b7d09d1) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:18.753071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753046c-5b70-4f95-8c1a-86a43216af87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:18.753192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8aa6a3-26e7-4229-9ed8-afbf4d85c4ac None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:18.754493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753046c-5b70-4f95-8c1a-86a43216af87 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:18.760365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 452/906] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:16:18 2022] GET /identity/v3/role_assignments?user.id=7c1a7d2677184c24ade81c397b7d09d1&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=ed3c392a05fc45baac04fa3092869141 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:18.761149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753046c-5b70-4f95-8c1a-86a43216af87 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6a8aa6a3-26e7-4229-9ed8-afbf4d85c4ac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:18.763029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:18.763972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:18.764905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 455/907] 10.222.0.60 () {56 vars in 1132 bytes} [Sat Sep 17 04:16:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:18.770528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aa6a3-26e7-4229-9ed8-afbf4d85c4ac None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:18.873168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753046c-5b70-4f95-8c1a-86a43216af87 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:18.886513 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aa6a3-26e7-4229-9ed8-afbf4d85c4ac None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:18.913360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c48e2c-d586-43c5-b683-e4156216f2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:18.913830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c48e2c-d586-43c5-b683-e4156216f2c4 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:18.914102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c48e2c-d586-43c5-b683-e4156216f2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:18.914367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c48e2c-d586-43c5-b683-e4156216f2c4 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:18.915076 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c48e2c-d586-43c5-b683-e4156216f2c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:18.916470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c48e2c-d586-43c5-b683-e4156216f2c4 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:18.927620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9808d4-fd28-4f6f-8b26-c86caff43b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:18.928662 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9808d4-fd28-4f6f-8b26-c86caff43b6a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:18.929284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9808d4-fd28-4f6f-8b26-c86caff43b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:18.929750 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9808d4-fd28-4f6f-8b26-c86caff43b6a None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:18.931001 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9808d4-fd28-4f6f-8b26-c86caff43b6a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:18.931647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9808d4-fd28-4f6f-8b26-c86caff43b6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:18.933410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9808d4-fd28-4f6f-8b26-c86caff43b6a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:18.936872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 453/908] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:16:18 2022] POST /identity/v3/services => generated 241 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:16:18.939669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 456/909] 10.222.0.60 () {56 vars in 1126 bytes} [Sat Sep 17 04:16:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:18.946244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c48e2c-d586-43c5-b683-e4156216f2c4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:19.004334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:19.067221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c48e2c-d586-43c5-b683-e4156216f2c4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:19.104660 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a602b46-e970-4014-96ae-179736af923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:19.105319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a602b46-e970-4014-96ae-179736af923f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:19.105596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a602b46-e970-4014-96ae-179736af923f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:19.105866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a602b46-e970-4014-96ae-179736af923f None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:19.106572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a602b46-e970-4014-96ae-179736af923f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:19.106878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a602b46-e970-4014-96ae-179736af923f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:19.108338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a602b46-e970-4014-96ae-179736af923f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:19.117861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 454/910] 10.222.0.60 () {56 vars in 1126 bytes} [Sat Sep 17 04:16:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:19.121711 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9808d4-fd28-4f6f-8b26-c86caff43b6a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:19.148651 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:19.231758 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9808d4-fd28-4f6f-8b26-c86caff43b6a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:19.267274 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc4229-9a74-4a09-be27-402666e45161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:19.267804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc4229-9a74-4a09-be27-402666e45161 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:19.268078 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc4229-9a74-4a09-be27-402666e45161 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:19.268384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc4229-9a74-4a09-be27-402666e45161 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:19.269122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc4229-9a74-4a09-be27-402666e45161 None admin] RBAC: Adding query filter params (interface=public, service_id=36136a50bb194523b4f573ddf911fcfe, region_id=RegionOne) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:19.269417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc4229-9a74-4a09-be27-402666e45161 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:19.270783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc4229-9a74-4a09-be27-402666e45161 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:19.280743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 457/911] 10.222.0.60 () {56 vars in 1251 bytes} [Sat Sep 17 04:16:19 2022] GET /identity/v3/endpoints?service_id=36136a50bb194523b4f573ddf911fcfe&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:19.282147 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a602b46-e970-4014-96ae-179736af923f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:19.386030 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a602b46-e970-4014-96ae-179736af923f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:19.396634 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:19.421319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc3393-7bf1-4eac-a525-9ff9be905b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:19.422329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc3393-7bf1-4eac-a525-9ff9be905b7c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:19.422604 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc3393-7bf1-4eac-a525-9ff9be905b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:19.422871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc3393-7bf1-4eac-a525-9ff9be905b7c None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:19.423474 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc3393-7bf1-4eac-a525-9ff9be905b7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:19.424838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc3393-7bf1-4eac-a525-9ff9be905b7c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:19.431195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 455/912] 10.222.0.60 () {56 vars in 1087 bytes} [Sat Sep 17 04:16:19 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:16:20.566177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3d24e-3b7c-40e1-b67d-3ba5005fa1d3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:20.566760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3d24e-3b7c-40e1-b67d-3ba5005fa1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:20.566760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3d24e-3b7c-40e1-b67d-3ba5005fa1d3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:20.568198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 458/913] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:20.573831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f72d2-5ad4-4465-be28-829dac8f88ea None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:20.574670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f72d2-5ad4-4465-be28-829dac8f88ea None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:20.575197 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f72d2-5ad4-4465-be28-829dac8f88ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:20.594455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-da8f72d2-5ad4-4465-be28-829dac8f88ea None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:20.596047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-da8f72d2-5ad4-4465-be28-829dac8f88ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:20.665592 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 456/914] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:20.932435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3d24e-3b7c-40e1-b67d-3ba5005fa1d3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:21.013206 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fa2b3-411e-4264-bc24-2aa4c5a2c57c None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.013484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fa2b3-411e-4264-bc24-2aa4c5a2c57c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.013758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fa2b3-411e-4264-bc24-2aa4c5a2c57c None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.015315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 457/915] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:21.020080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6283c68-df46-4434-a4d3-46aeff5b9ddb None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.020365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6283c68-df46-4434-a4d3-46aeff5b9ddb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.020637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6283c68-df46-4434-a4d3-46aeff5b9ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.022830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:21.045897 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e6283c68-df46-4434-a4d3-46aeff5b9ddb None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:21.046187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3d24e-3b7c-40e1-b67d-3ba5005fa1d3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:21.047076 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e6283c68-df46-4434-a4d3-46aeff5b9ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:21.083039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f3aa1-bb3c-45d8-aeb5-4b7f9682f879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:21.083980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f3aa1-bb3c-45d8-aeb5-4b7f9682f879 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.084399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f3aa1-bb3c-45d8-aeb5-4b7f9682f879 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.084971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f3aa1-bb3c-45d8-aeb5-4b7f9682f879 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.085942 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f3aa1-bb3c-45d8-aeb5-4b7f9682f879 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:21.087521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f3aa1-bb3c-45d8-aeb5-4b7f9682f879 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-325f3aa1-bb3c-45d8-aeb5-4b7f9682f879 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:21.097332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:21.098579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 04:16:21.099527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:21.102014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 459/916] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:16:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:21.107865 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f3aa1-bb3c-45d8-aeb5-4b7f9682f879 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:21.133133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 458/917] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:16:21.216813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f3aa1-bb3c-45d8-aeb5-4b7f9682f879 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:21.251257 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefac26-2ab4-4ca6-8d1f-c3c6ff12b344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:21.251869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefac26-2ab4-4ca6-8d1f-c3c6ff12b344 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.252138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefac26-2ab4-4ca6-8d1f-c3c6ff12b344 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.252401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefac26-2ab4-4ca6-8d1f-c3c6ff12b344 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.253031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefac26-2ab4-4ca6-8d1f-c3c6ff12b344 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:21.253351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefac26-2ab4-4ca6-8d1f-c3c6ff12b344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:21.254854 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefac26-2ab4-4ca6-8d1f-c3c6ff12b344 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:21.255669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9f479-a60b-470d-8969-c5ae2b89d71b None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.255955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9f479-a60b-470d-8969-c5ae2b89d71b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.256229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9f479-a60b-470d-8969-c5ae2b89d71b None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.257859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 459/918] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:16:21.260128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 460/919] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:16:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:21.265782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefac26-2ab4-4ca6-8d1f-c3c6ff12b344 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:21.266007 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87058acd-7588-47ea-bc5f-0c644c9705ac None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.266289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87058acd-7588-47ea-bc5f-0c644c9705ac None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.266562 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87058acd-7588-47ea-bc5f-0c644c9705ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.282231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:21.288196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-87058acd-7588-47ea-bc5f-0c644c9705ac None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:21.291241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-87058acd-7588-47ea-bc5f-0c644c9705ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:21.321298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:21.385132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 460/920] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:21.385354 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefac26-2ab4-4ca6-8d1f-c3c6ff12b344 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:21.407431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87058acd-7588-47ea-bc5f-0c644c9705ac None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:21.420779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c666c3d-f2ff-450c-bd20-cdf121ded190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:21.421426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c666c3d-f2ff-450c-bd20-cdf121ded190 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.421703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c666c3d-f2ff-450c-bd20-cdf121ded190 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.422213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c666c3d-f2ff-450c-bd20-cdf121ded190 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.422962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c666c3d-f2ff-450c-bd20-cdf121ded190 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:21.423263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c666c3d-f2ff-450c-bd20-cdf121ded190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:21.424944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c666c3d-f2ff-450c-bd20-cdf121ded190 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:21.432397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 461/921] 10.222.0.60 () {56 vars in 1114 bytes} [Sat Sep 17 04:16:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:21.438115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c666c3d-f2ff-450c-bd20-cdf121ded190 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:21.512523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87058acd-7588-47ea-bc5f-0c644c9705ac None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:21.525388 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:21.549277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f1b18-29e1-42bb-849f-a4145e9e44b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:21.550285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f1b18-29e1-42bb-849f-a4145e9e44b3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.550497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c666c3d-f2ff-450c-bd20-cdf121ded190 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:21.550972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f1b18-29e1-42bb-849f-a4145e9e44b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.551437 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f1b18-29e1-42bb-849f-a4145e9e44b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.562101 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f1b18-29e1-42bb-849f-a4145e9e44b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-f07f1b18-29e1-42bb-849f-a4145e9e44b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:21.567226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:21.568140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:21.568968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:21.568968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:21.568968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:21.568968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:21.568968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:21.568968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:21.568968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:21.568968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:21.568968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:21.568968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:21.568968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 04:16:21.568968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:21.576543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 461/922] 10.222.0.60 () {56 vars in 1120 bytes} [Sat Sep 17 04:16:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:21.578114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f1b18-29e1-42bb-849f-a4145e9e44b3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:21.586674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2b3e0-c8a0-4107-b304-5f2ded847356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:21.587321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2b3e0-c8a0-4107-b304-5f2ded847356 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.587591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2b3e0-c8a0-4107-b304-5f2ded847356 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.587853 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2b3e0-c8a0-4107-b304-5f2ded847356 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.588568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2b3e0-c8a0-4107-b304-5f2ded847356 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:21.590096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2b3e0-c8a0-4107-b304-5f2ded847356 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:21.607563 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:21.618310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 462/923] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:16:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:16:21.620510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc32702-6b92-4820-ae27-7cd7891de043 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.620839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc32702-6b92-4820-ae27-7cd7891de043 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.621121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc32702-6b92-4820-ae27-7cd7891de043 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.622493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 463/924] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:21.628827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43cabf-0c10-43d0-bff7-e22b2dec2d19 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.629239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43cabf-0c10-43d0-bff7-e22b2dec2d19 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.629406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43cabf-0c10-43d0-bff7-e22b2dec2d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.651421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2e43cabf-0c10-43d0-bff7-e22b2dec2d19 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:21.653059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2e43cabf-0c10-43d0-bff7-e22b2dec2d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:21.701392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f1b18-29e1-42bb-849f-a4145e9e44b3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:21.752150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 464/925] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:21.753351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43cabf-0c10-43d0-bff7-e22b2dec2d19 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:21.755007 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fef898-2301-4ed5-9637-f286e7652022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:21.755943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fef898-2301-4ed5-9637-f286e7652022 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.756449 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fef898-2301-4ed5-9637-f286e7652022 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.756920 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fef898-2301-4ed5-9637-f286e7652022 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.757901 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fef898-2301-4ed5-9637-f286e7652022 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:21.758409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fef898-2301-4ed5-9637-f286e7652022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:21.760146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fef898-2301-4ed5-9637-f286e7652022 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:21.770294 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 462/926] 10.222.0.60 () {56 vars in 1115 bytes} [Sat Sep 17 04:16:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:21.776141 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fef898-2301-4ed5-9637-f286e7652022 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:21.831978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:21.876053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:21.880691 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43cabf-0c10-43d0-bff7-e22b2dec2d19 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:21.896432 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fef898-2301-4ed5-9637-f286e7652022 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:21.925903 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec954f9-9d3e-45d9-b64d-75ca7310b946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:21.926764 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec954f9-9d3e-45d9-b64d-75ca7310b946 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.927110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec954f9-9d3e-45d9-b64d-75ca7310b946 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.927453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec954f9-9d3e-45d9-b64d-75ca7310b946 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.928171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec954f9-9d3e-45d9-b64d-75ca7310b946 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:21.930074 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec954f9-9d3e-45d9-b64d-75ca7310b946 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:21.930977 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd8cdd-dbce-4087-9cb6-6ba59ba4b771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:21.931806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd8cdd-dbce-4087-9cb6-6ba59ba4b771 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:21.932253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd8cdd-dbce-4087-9cb6-6ba59ba4b771 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:21.932708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd8cdd-dbce-4087-9cb6-6ba59ba4b771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-dec954f9-9d3e-45d9-b64d-75ca7310b946 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:21.940644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:21.941671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:21.942787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 465/927] 10.222.0.60 () {56 vars in 1126 bytes} [Sat Sep 17 04:16:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:21.947603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd8cdd-dbce-4087-9cb6-6ba59ba4b771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:21.947944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec954f9-9d3e-45d9-b64d-75ca7310b946 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:21.949528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd8cdd-dbce-4087-9cb6-6ba59ba4b771 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-58dd8cdd-dbce-4087-9cb6-6ba59ba4b771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:21.964234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:21.965265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:21.966138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:21.966138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:21.966138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:21.968224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 463/928] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:21.973886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd8cdd-dbce-4087-9cb6-6ba59ba4b771 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:22.060790 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec954f9-9d3e-45d9-b64d-75ca7310b946 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:22.087726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:22.089794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd8cdd-dbce-4087-9cb6-6ba59ba4b771 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:22.099265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620750d-2bd0-42ee-9fc5-6bf6a026d645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:22.099887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620750d-2bd0-42ee-9fc5-6bf6a026d645 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:22.100159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620750d-2bd0-42ee-9fc5-6bf6a026d645 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:22.100425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620750d-2bd0-42ee-9fc5-6bf6a026d645 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:22.101053 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620750d-2bd0-42ee-9fc5-6bf6a026d645 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:22.101377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620750d-2bd0-42ee-9fc5-6bf6a026d645 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:22.102779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620750d-2bd0-42ee-9fc5-6bf6a026d645 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:22.108350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 466/929] 10.222.0.60 () {56 vars in 1122 bytes} [Sat Sep 17 04:16:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:22.109220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620750d-2bd0-42ee-9fc5-6bf6a026d645 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:22.130169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf0e9e-37a9-4846-95e5-5d79de61499e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:22.130773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf0e9e-37a9-4846-95e5-5d79de61499e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:22.131045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf0e9e-37a9-4846-95e5-5d79de61499e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:22.131421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf0e9e-37a9-4846-95e5-5d79de61499e None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:22.132057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf0e9e-37a9-4846-95e5-5d79de61499e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:22.132353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf0e9e-37a9-4846-95e5-5d79de61499e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:22.133752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf0e9e-37a9-4846-95e5-5d79de61499e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:22.135499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:22.147823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 464/930] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:22.148647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf0e9e-37a9-4846-95e5-5d79de61499e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:22.214735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620750d-2bd0-42ee-9fc5-6bf6a026d645 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:22.249173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf0e9e-37a9-4846-95e5-5d79de61499e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:22.250652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6dc734-81a0-4fb0-b1cc-66a3753ecfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:22.251338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6dc734-81a0-4fb0-b1cc-66a3753ecfef None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:22.251617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6dc734-81a0-4fb0-b1cc-66a3753ecfef None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:22.251880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6dc734-81a0-4fb0-b1cc-66a3753ecfef None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:22.252495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6dc734-81a0-4fb0-b1cc-66a3753ecfef None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:22.253921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6dc734-81a0-4fb0-b1cc-66a3753ecfef None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ee6dc734-81a0-4fb0-b1cc-66a3753ecfef None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:22.262684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:22.263634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:22.264523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 467/931] 10.222.0.60 () {56 vars in 1132 bytes} [Sat Sep 17 04:16:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:22.265346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6dc734-81a0-4fb0-b1cc-66a3753ecfef None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:22.291213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e5477-d351-4750-9278-527af7dc3f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:22.291825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e5477-d351-4750-9278-527af7dc3f0c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:22.292095 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e5477-d351-4750-9278-527af7dc3f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:22.292357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e5477-d351-4750-9278-527af7dc3f0c None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:22.292980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e5477-d351-4750-9278-527af7dc3f0c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:22.293381 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e5477-d351-4750-9278-527af7dc3f0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:22.294740 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e5477-d351-4750-9278-527af7dc3f0c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:22.299768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 465/932] 10.222.0.60 () {56 vars in 1122 bytes} [Sat Sep 17 04:16:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:22.300598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e5477-d351-4750-9278-527af7dc3f0c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:22.328800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:22.363860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:22.381556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6dc734-81a0-4fb0-b1cc-66a3753ecfef None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:22.407993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e5477-d351-4750-9278-527af7dc3f0c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:22.414631 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5acda4-4a73-4249-918d-49cde8f72ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:22.415375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5acda4-4a73-4249-918d-49cde8f72ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:22.415651 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5acda4-4a73-4249-918d-49cde8f72ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:22.415916 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5acda4-4a73-4249-918d-49cde8f72ed9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:22.423368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5acda4-4a73-4249-918d-49cde8f72ed9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:22.424808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5acda4-4a73-4249-918d-49cde8f72ed9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3b5acda4-4a73-4249-918d-49cde8f72ed9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:22.435979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:22.437045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:22.438059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:22.438059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:22.438059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:22.438059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:22.438059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:22.438059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:22.438059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:22.438059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:22.438059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 04:16:22.438059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:22.438059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 468/933] 10.222.0.60 () {56 vars in 1117 bytes} [Sat Sep 17 04:16:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:22.438059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5acda4-4a73-4249-918d-49cde8f72ed9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:22.450096 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e879b-d8d3-47aa-9946-08233bb7df04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:22.451001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e879b-d8d3-47aa-9946-08233bb7df04 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:22.451451 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e879b-d8d3-47aa-9946-08233bb7df04 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:22.451906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e879b-d8d3-47aa-9946-08233bb7df04 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:22.452705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e879b-d8d3-47aa-9946-08233bb7df04 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:22.453234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e879b-d8d3-47aa-9946-08233bb7df04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:22.454804 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e879b-d8d3-47aa-9946-08233bb7df04 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:22.460510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 466/934] 10.222.0.60 () {56 vars in 1126 bytes} [Sat Sep 17 04:16:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:22.461572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e879b-d8d3-47aa-9946-08233bb7df04 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:22.544151 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5acda4-4a73-4249-918d-49cde8f72ed9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:22.556755 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:22.569448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e879b-d8d3-47aa-9946-08233bb7df04 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:22.578014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9ab1f-5030-4727-a104-8ba438d47d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:22.578622 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9ab1f-5030-4727-a104-8ba438d47d1b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:22.578947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9ab1f-5030-4727-a104-8ba438d47d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:22.579154 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9ab1f-5030-4727-a104-8ba438d47d1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:22.579849 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9ab1f-5030-4727-a104-8ba438d47d1b None admin] RBAC: Adding query filter params (interface=public, service_id=909173668f2a4a97abe351e0a0f3d5cd, region_id=RegionOne) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:22.580159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9ab1f-5030-4727-a104-8ba438d47d1b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:22.581535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9ab1f-5030-4727-a104-8ba438d47d1b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:22.586487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 469/935] 10.222.0.60 () {56 vars in 1251 bytes} [Sat Sep 17 04:16:22 2022] GET /identity/v3/endpoints?service_id=909173668f2a4a97abe351e0a0f3d5cd&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:22.587401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9ab1f-5030-4727-a104-8ba438d47d1b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:22.601623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:22.615134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b2c05-e838-494e-a528-04b1540046d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:22.616376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b2c05-e838-494e-a528-04b1540046d5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:22.616915 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b2c05-e838-494e-a528-04b1540046d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:22.617228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b2c05-e838-494e-a528-04b1540046d5 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:22.618148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b2c05-e838-494e-a528-04b1540046d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:22.618454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b2c05-e838-494e-a528-04b1540046d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:22.619986 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b2c05-e838-494e-a528-04b1540046d5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:22.642369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 467/936] 10.222.0.60 () {56 vars in 1149 bytes} [Sat Sep 17 04:16:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:22.642369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b2c05-e838-494e-a528-04b1540046d5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:22.701201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9ab1f-5030-4727-a104-8ba438d47d1b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:22.734698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42ea28-ffb8-4761-986d-7db1cd53d4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:22.735377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42ea28-ffb8-4761-986d-7db1cd53d4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:22.735650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42ea28-ffb8-4761-986d-7db1cd53d4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:22.735917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42ea28-ffb8-4761-986d-7db1cd53d4c9 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:22.736536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42ea28-ffb8-4761-986d-7db1cd53d4c9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:22.736827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42ea28-ffb8-4761-986d-7db1cd53d4c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:22.738225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42ea28-ffb8-4761-986d-7db1cd53d4c9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:22.743566 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 470/937] 10.222.0.60 () {56 vars in 1126 bytes} [Sat Sep 17 04:16:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:22.744331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42ea28-ffb8-4761-986d-7db1cd53d4c9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:22.757913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b2c05-e838-494e-a528-04b1540046d5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:22.791284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:22.793970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc39a30-fcf0-460b-ab0f-189d204a77af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:22.794829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc39a30-fcf0-460b-ab0f-189d204a77af None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:22.795101 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc39a30-fcf0-460b-ab0f-189d204a77af None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:22.795367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc39a30-fcf0-460b-ab0f-189d204a77af None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:22.796046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc39a30-fcf0-460b-ab0f-189d204a77af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:22.797433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc39a30-fcf0-460b-ab0f-189d204a77af None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:22.803619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 468/938] 10.222.0.60 () {56 vars in 1087 bytes} [Sat Sep 17 04:16:22 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:16:22.804218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc39a30-fcf0-460b-ab0f-189d204a77af None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:22.865607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:22.878737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42ea28-ffb8-4761-986d-7db1cd53d4c9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:22.924349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6181d-8639-4971-96a4-64ea7f116768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:22.926021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6181d-8639-4971-96a4-64ea7f116768 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:22.926802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6181d-8639-4971-96a4-64ea7f116768 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:22.927268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6181d-8639-4971-96a4-64ea7f116768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:22.939310 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc39a30-fcf0-460b-ab0f-189d204a77af None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:22.945003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6181d-8639-4971-96a4-64ea7f116768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:22.946963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6181d-8639-4971-96a4-64ea7f116768 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-97f6181d-8639-4971-96a4-64ea7f116768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:22.965923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:22.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:22.967876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:22.967876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:22.967876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:22.967876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 471/939] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:22.974145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6181d-8639-4971-96a4-64ea7f116768 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:22.983286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249b2bf-2d0c-4195-87b3-e63ae4abf03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:22.984179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249b2bf-2d0c-4195-87b3-e63ae4abf03f None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:22.984622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249b2bf-2d0c-4195-87b3-e63ae4abf03f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:22.985130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249b2bf-2d0c-4195-87b3-e63ae4abf03f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:22.985977 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249b2bf-2d0c-4195-87b3-e63ae4abf03f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:22.987538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249b2bf-2d0c-4195-87b3-e63ae4abf03f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:23.012176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 469/940] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:16:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:16:23.090935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:23.102174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6181d-8639-4971-96a4-64ea7f116768 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:23.148126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86563ef1-b3dd-4c94-88cc-9e0a19364f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:23.149144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86563ef1-b3dd-4c94-88cc-9e0a19364f75 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:23.149668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86563ef1-b3dd-4c94-88cc-9e0a19364f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:23.150153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86563ef1-b3dd-4c94-88cc-9e0a19364f75 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:23.151024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86563ef1-b3dd-4c94-88cc-9e0a19364f75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:23.151535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86563ef1-b3dd-4c94-88cc-9e0a19364f75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:23.153236 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86563ef1-b3dd-4c94-88cc-9e0a19364f75 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:23.168087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 472/941] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:23.173713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249b2bf-2d0c-4195-87b3-e63ae4abf03f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:23.272512 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:23.275943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249b2bf-2d0c-4195-87b3-e63ae4abf03f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:23.309959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb95b8c-1890-4e4c-be89-d1ea9e28bcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:23.310694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb95b8c-1890-4e4c-be89-d1ea9e28bcdd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:23.310976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb95b8c-1890-4e4c-be89-d1ea9e28bcdd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:23.311236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb95b8c-1890-4e4c-be89-d1ea9e28bcdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:23.318395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb95b8c-1890-4e4c-be89-d1ea9e28bcdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:23.319834 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb95b8c-1890-4e4c-be89-d1ea9e28bcdd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-7eb95b8c-1890-4e4c-be89-d1ea9e28bcdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:23.331277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:23.332193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:23.333097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:23.333097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:23.333097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:23.333097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:23.333097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:23.333097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:23.333097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:23.333097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:23.333097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 470/942] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:23.338632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86563ef1-b3dd-4c94-88cc-9e0a19364f75 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:23.450169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86563ef1-b3dd-4c94-88cc-9e0a19364f75 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:23.486905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0c24c-4d0a-45c9-8a2f-a365b76bfe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:23.487909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0c24c-4d0a-45c9-8a2f-a365b76bfe4a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:23.488393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0c24c-4d0a-45c9-8a2f-a365b76bfe4a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:23.488873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0c24c-4d0a-45c9-8a2f-a365b76bfe4a None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:23.489782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0c24c-4d0a-45c9-8a2f-a365b76bfe4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:23.490310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0c24c-4d0a-45c9-8a2f-a365b76bfe4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:23.492287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0c24c-4d0a-45c9-8a2f-a365b76bfe4a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:23.504924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 473/943] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:23.513101 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb95b8c-1890-4e4c-be89-d1ea9e28bcdd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:23.614622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb95b8c-1890-4e4c-be89-d1ea9e28bcdd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:23.647817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153df073-7c30-4cf8-bae6-dc6fffa85c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:23.648488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153df073-7c30-4cf8-bae6-dc6fffa85c48 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:23.648750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153df073-7c30-4cf8-bae6-dc6fffa85c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:23.649023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153df073-7c30-4cf8-bae6-dc6fffa85c48 None admin] PATH_INFO: `/v3/projects/58466195aca94b188d82c1b3a647e767/users/7c1a7d2677184c24ade81c397b7d09d1/roles/ed3c392a05fc45baac04fa3092869141` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:23.651265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153df073-7c30-4cf8-bae6-dc6fffa85c48 None admin] RBAC: Authorizing `identity:create_grant(project_id=58466195aca94b188d82c1b3a647e767, user_id=7c1a7d2677184c24ade81c397b7d09d1, role_id=ed3c392a05fc45baac04fa3092869141)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:23.652741 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153df073-7c30-4cf8-bae6-dc6fffa85c48 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:23.677241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 471/944] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:16:23 2022] PUT /identity/v3/projects/58466195aca94b188d82c1b3a647e767/users/7c1a7d2677184c24ade81c397b7d09d1/roles/ed3c392a05fc45baac04fa3092869141 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:16:23.880242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f5fe9-0101-4ba5-8269-137a18f14e6b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:23.880776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f5fe9-0101-4ba5-8269-137a18f14e6b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:23.880776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f5fe9-0101-4ba5-8269-137a18f14e6b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:23.882289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 474/945] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:23.888146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47228d35-84b8-4342-9be5-4bd2ab9d3bd5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:23.888431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47228d35-84b8-4342-9be5-4bd2ab9d3bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:23.888700 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47228d35-84b8-4342-9be5-4bd2ab9d3bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:23.908769 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-47228d35-84b8-4342-9be5-4bd2ab9d3bd5 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:23.909987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-47228d35-84b8-4342-9be5-4bd2ab9d3bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:23.924840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:23.979029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 472/946] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:24.247273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f5fe9-0101-4ba5-8269-137a18f14e6b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:24.259098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:24.351582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f5fe9-0101-4ba5-8269-137a18f14e6b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:24.385890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61b31a-9357-40fb-a6fb-62bbe52a2c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:24.386863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61b31a-9357-40fb-a6fb-62bbe52a2c16 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:24.387336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61b31a-9357-40fb-a6fb-62bbe52a2c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:24.387865 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61b31a-9357-40fb-a6fb-62bbe52a2c16 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:24.388732 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61b31a-9357-40fb-a6fb-62bbe52a2c16 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:24.390397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61b31a-9357-40fb-a6fb-62bbe52a2c16 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:24.393247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: WARNING keystone.server.flask.common [None req-0b61b31a-9357-40fb-a6fb-62bbe52a2c16 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 17 04:16:24.414033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.sql.core [None req-0b61b31a-9357-40fb-a6fb-62bbe52a2c16 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 17 04:16:24.414033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 17 04:16:24.414033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [parameters: {'id': '2ecc09e65d6b4980adb00b350309d85d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 17 04:16:24.414033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81417) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0b61b31a-9357-40fb-a6fb-62bbe52a2c16 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2ecc09e65d6b4980adb00b350309d85d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2ecc09e65d6b4980adb00b350309d85d. Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application conn.query(q) Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 04:16:24.422095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result.read() Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 17 04:16:24.422892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application next(self.gen) Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application yield resource Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application next(self.gen) Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise self.value Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application session.commit() Sep 17 04:16:24.423824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.session.flush() Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application compat.raise_( Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 04:16:24.424703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 04:16:24.425626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application conn.query(q) Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result.read() Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 04:16:24.426429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [parameters: {'id': '2ecc09e65d6b4980adb00b350309d85d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:24.427276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:24.429243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:24.431893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:24.431893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 17 04:16:24.431893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 17 04:16:24.431893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:24.431893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:24.431893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 17 04:16:24.431893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 17 04:16:24.431893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2ecc09e65d6b4980adb00b350309d85d. Sep 17 04:16:24.431893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:24.431893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 475/947] 10.222.0.60 () {60 vars in 1140 bytes} [Sat Sep 17 04:16:24 2022] POST /identity/v3/projects => generated 262 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:24.436614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47228d35-84b8-4342-9be5-4bd2ab9d3bd5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:24.535945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47228d35-84b8-4342-9be5-4bd2ab9d3bd5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:24.570711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc03c29-d8fa-413d-a0d4-e62dc9db5d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:24.571770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc03c29-d8fa-413d-a0d4-e62dc9db5d69 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:24.572251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc03c29-d8fa-413d-a0d4-e62dc9db5d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:24.572747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc03c29-d8fa-413d-a0d4-e62dc9db5d69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:24.579561 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc03c29-d8fa-413d-a0d4-e62dc9db5d69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:24.581246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc03c29-d8fa-413d-a0d4-e62dc9db5d69 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-4dc03c29-d8fa-413d-a0d4-e62dc9db5d69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:24.592557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:24.593481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:24.594377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:24.594377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:24.594377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:24.594377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:24.594377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:24.594377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:24.594377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:24.594377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:24.596748 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 473/948] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:24.602641 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61b31a-9357-40fb-a6fb-62bbe52a2c16 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:24.689581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:24.711617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61b31a-9357-40fb-a6fb-62bbe52a2c16 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:24.745570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c6bed-ccbb-42d0-9db0-0ecadb347a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:24.746356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c6bed-ccbb-42d0-9db0-0ecadb347a5b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:24.746729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c6bed-ccbb-42d0-9db0-0ecadb347a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:24.747094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c6bed-ccbb-42d0-9db0-0ecadb347a5b None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:24.747856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c6bed-ccbb-42d0-9db0-0ecadb347a5b None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:24.748252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c6bed-ccbb-42d0-9db0-0ecadb347a5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:24.750098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c6bed-ccbb-42d0-9db0-0ecadb347a5b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:24.772666 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 476/949] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:16:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:25.148109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762f75b-756f-461d-b469-1178ca17a0bf None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:25.148651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762f75b-756f-461d-b469-1178ca17a0bf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:25.148651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762f75b-756f-461d-b469-1178ca17a0bf None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:25.150341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 474/950] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:25.155759 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d157c-bc14-49ca-9499-9c0eb2e4a551 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:25.156189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d157c-bc14-49ca-9499-9c0eb2e4a551 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:25.156360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d157c-bc14-49ca-9499-9c0eb2e4a551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:25.175401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ec6d157c-bc14-49ca-9499-9c0eb2e4a551 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:25.176910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ec6d157c-bc14-49ca-9499-9c0eb2e4a551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:25.251089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 477/951] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:25.314080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e6b35-8591-49c6-9b37-4586fd8ae17f None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:25.314739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e6b35-8591-49c6-9b37-4586fd8ae17f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:25.315253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e6b35-8591-49c6-9b37-4586fd8ae17f None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:25.317042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 475/952] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:25.322630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bac761-f1de-4db6-9c9b-a2878b9d1d9d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:25.323141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bac761-f1de-4db6-9c9b-a2878b9d1d9d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:25.323650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bac761-f1de-4db6-9c9b-a2878b9d1d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:25.343085 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f1bac761-f1de-4db6-9c9b-a2878b9d1d9d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:25.344653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f1bac761-f1de-4db6-9c9b-a2878b9d1d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:25.400076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:25.419157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 478/953] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:25.516820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e6b35-8591-49c6-9b37-4586fd8ae17f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:25.529325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:25.618040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e6b35-8591-49c6-9b37-4586fd8ae17f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:25.651586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202ff4f-3293-4198-b0e4-1bf6ed44b808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:25.652145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202ff4f-3293-4198-b0e4-1bf6ed44b808 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:25.652414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202ff4f-3293-4198-b0e4-1bf6ed44b808 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:25.652678 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202ff4f-3293-4198-b0e4-1bf6ed44b808 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:25.653309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202ff4f-3293-4198-b0e4-1bf6ed44b808 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:25.654733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202ff4f-3293-4198-b0e4-1bf6ed44b808 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c202ff4f-3293-4198-b0e4-1bf6ed44b808 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:25.674046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:25.675256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:25.676149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 476/954] 10.222.0.60 () {56 vars in 1126 bytes} [Sat Sep 17 04:16:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:25.681683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bac761-f1de-4db6-9c9b-a2878b9d1d9d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:25.698656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202ff4f-3293-4198-b0e4-1bf6ed44b808 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:25.779847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:25.815270 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bac761-f1de-4db6-9c9b-a2878b9d1d9d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:25.817699 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202ff4f-3293-4198-b0e4-1bf6ed44b808 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:25.862248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c366b1-5f66-44df-ab19-2b40b29bba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:25.862866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c366b1-5f66-44df-ab19-2b40b29bba51 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:25.863214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c366b1-5f66-44df-ab19-2b40b29bba51 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:25.863501 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c366b1-5f66-44df-ab19-2b40b29bba51 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:25.864137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c366b1-5f66-44df-ab19-2b40b29bba51 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:25.864440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c366b1-5f66-44df-ab19-2b40b29bba51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:25.864707 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed91cee-3dfa-46d2-b5cb-ecd770af077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:25.865587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed91cee-3dfa-46d2-b5cb-ecd770af077f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:25.866203 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c366b1-5f66-44df-ab19-2b40b29bba51 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:25.866598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed91cee-3dfa-46d2-b5cb-ecd770af077f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:25.867069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed91cee-3dfa-46d2-b5cb-ecd770af077f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:25.868555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed91cee-3dfa-46d2-b5cb-ecd770af077f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:25.870438 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed91cee-3dfa-46d2-b5cb-ecd770af077f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:25.871877 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 479/955] 10.222.0.60 () {56 vars in 1122 bytes} [Sat Sep 17 04:16:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:25.877145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c366b1-5f66-44df-ab19-2b40b29bba51 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-aed91cee-3dfa-46d2-b5cb-ecd770af077f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:25.883850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:25.884891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 04:16:25.886084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:25.888319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 477/956] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:25.894457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed91cee-3dfa-46d2-b5cb-ecd770af077f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:25.934245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:25.985659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c366b1-5f66-44df-ab19-2b40b29bba51 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:25.997715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed91cee-3dfa-46d2-b5cb-ecd770af077f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:26.023548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2512d-bac5-45ed-b51c-351aa271f5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:26.024171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2512d-bac5-45ed-b51c-351aa271f5ee None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:26.024447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2512d-bac5-45ed-b51c-351aa271f5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:26.024709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2512d-bac5-45ed-b51c-351aa271f5ee None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:26.025362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2512d-bac5-45ed-b51c-351aa271f5ee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:26.025656 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2512d-bac5-45ed-b51c-351aa271f5ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:26.027020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2512d-bac5-45ed-b51c-351aa271f5ee None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:26.031777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 480/957] 10.222.0.60 () {56 vars in 1122 bytes} [Sat Sep 17 04:16:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:26.032908 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88193b03-71c1-4cf6-9c19-f4af585d1ead None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:26.033218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88193b03-71c1-4cf6-9c19-f4af585d1ead None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:26.033488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88193b03-71c1-4cf6-9c19-f4af585d1ead None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:26.034816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 481/958] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:26.037597 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88193b03-71c1-4cf6-9c19-f4af585d1ead None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:26.040175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e13699-cf8b-43aa-91b5-aca1debfcaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:26.040796 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e13699-cf8b-43aa-91b5-aca1debfcaac None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:26.041497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e13699-cf8b-43aa-91b5-aca1debfcaac None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:26.041776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e13699-cf8b-43aa-91b5-aca1debfcaac None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:26.042411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e13699-cf8b-43aa-91b5-aca1debfcaac None admin] RBAC: Adding query filter params (name=compute) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:26.042790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e13699-cf8b-43aa-91b5-aca1debfcaac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:26.044590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e13699-cf8b-43aa-91b5-aca1debfcaac None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:26.049330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 478/959] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:16:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:26.050268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2b3de-fc76-4849-9ed7-4d2f1ed71654 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:26.050551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2b3de-fc76-4849-9ed7-4d2f1ed71654 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:26.050904 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2b3de-fc76-4849-9ed7-4d2f1ed71654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:26.057201 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:26.069199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-83a2b3de-fc76-4849-9ed7-4d2f1ed71654 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:26.070373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-83a2b3de-fc76-4849-9ed7-4d2f1ed71654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:26.137000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 479/960] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:26.146049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2b3de-fc76-4849-9ed7-4d2f1ed71654 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:26.160710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88193b03-71c1-4cf6-9c19-f4af585d1ead None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:26.175649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:26.200040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef7fbd-0b7b-4768-8483-c9e2e69b333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:26.200955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef7fbd-0b7b-4768-8483-c9e2e69b333b None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:26.201485 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef7fbd-0b7b-4768-8483-c9e2e69b333b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:26.201966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef7fbd-0b7b-4768-8483-c9e2e69b333b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:26.202827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef7fbd-0b7b-4768-8483-c9e2e69b333b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:26.204406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef7fbd-0b7b-4768-8483-c9e2e69b333b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:26.231225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 482/961] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:16:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:16:26.274292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2b3de-fc76-4849-9ed7-4d2f1ed71654 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:26.308589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:26.321006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3f551-0970-44e1-bf6d-7a73b6f257de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:26.322374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3f551-0970-44e1-bf6d-7a73b6f257de None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:26.322655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3f551-0970-44e1-bf6d-7a73b6f257de None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:26.322923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3f551-0970-44e1-bf6d-7a73b6f257de None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:26.323626 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3f551-0970-44e1-bf6d-7a73b6f257de None admin] RBAC: Adding query filter params (type=compute) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:26.323934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3f551-0970-44e1-bf6d-7a73b6f257de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:26.325363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3f551-0970-44e1-bf6d-7a73b6f257de None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:26.331118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 480/962] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:16:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:26.423419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef7fbd-0b7b-4768-8483-c9e2e69b333b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:26.535149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef7fbd-0b7b-4768-8483-c9e2e69b333b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:26.572493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b75c16-44e3-4fe8-a062-59ab245dc1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:26.573160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b75c16-44e3-4fe8-a062-59ab245dc1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:26.573487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b75c16-44e3-4fe8-a062-59ab245dc1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:26.573760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b75c16-44e3-4fe8-a062-59ab245dc1a6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:26.584504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b75c16-44e3-4fe8-a062-59ab245dc1a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e6b75c16-44e3-4fe8-a062-59ab245dc1a6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:26.589845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:26.590802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:26.591649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 483/963] 10.222.0.60 () {56 vars in 1120 bytes} [Sat Sep 17 04:16:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:26.597386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3f551-0970-44e1-bf6d-7a73b6f257de None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:26.702017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3f551-0970-44e1-bf6d-7a73b6f257de None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:26.737484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df6d0d-2e2d-4781-8a10-af913b829f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:26.738107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df6d0d-2e2d-4781-8a10-af913b829f4d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:26.738385 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df6d0d-2e2d-4781-8a10-af913b829f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:26.738664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df6d0d-2e2d-4781-8a10-af913b829f4d None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:26.739337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df6d0d-2e2d-4781-8a10-af913b829f4d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:26.739639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df6d0d-2e2d-4781-8a10-af913b829f4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:26.742301 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df6d0d-2e2d-4781-8a10-af913b829f4d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:26.751159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 481/964] 10.222.0.60 () {56 vars in 1115 bytes} [Sat Sep 17 04:16:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:26.757047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b75c16-44e3-4fe8-a062-59ab245dc1a6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:26.787023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:26.864580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b75c16-44e3-4fe8-a062-59ab245dc1a6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:26.899060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484065af-dd01-4410-99b0-dba0a20d8e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:26.899733 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484065af-dd01-4410-99b0-dba0a20d8e3a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:26.900005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484065af-dd01-4410-99b0-dba0a20d8e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:26.900274 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484065af-dd01-4410-99b0-dba0a20d8e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:26.910860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484065af-dd01-4410-99b0-dba0a20d8e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:26.912316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484065af-dd01-4410-99b0-dba0a20d8e3a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-484065af-dd01-4410-99b0-dba0a20d8e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:26.927403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:26.928386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:26.929314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:26.929314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:26.929314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:26.929314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 484/965] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:26.935092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df6d0d-2e2d-4781-8a10-af913b829f4d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:27.001547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:27.044086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df6d0d-2e2d-4781-8a10-af913b829f4d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:27.078175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68352d8d-8810-422d-8b9b-241be0b5fb68 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:27.078373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9468a-09e4-42f9-80ae-46a9b94bf58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:27.078950 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9468a-09e4-42f9-80ae-46a9b94bf58b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:27.079178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68352d8d-8810-422d-8b9b-241be0b5fb68 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:27.079279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9468a-09e4-42f9-80ae-46a9b94bf58b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:27.079476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9468a-09e4-42f9-80ae-46a9b94bf58b None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:27.079935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68352d8d-8810-422d-8b9b-241be0b5fb68 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:27.080116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9468a-09e4-42f9-80ae-46a9b94bf58b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:27.080485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9468a-09e4-42f9-80ae-46a9b94bf58b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:27.081959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9468a-09e4-42f9-80ae-46a9b94bf58b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:27.082380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 485/966] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:27.087942 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89182b5-ed91-4774-9d78-2c703b4c6748 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:27.088504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89182b5-ed91-4774-9d78-2c703b4c6748 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:27.088975 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89182b5-ed91-4774-9d78-2c703b4c6748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:27.095130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 482/967] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:27.100676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9468a-09e4-42f9-80ae-46a9b94bf58b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:27.108620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f89182b5-ed91-4774-9d78-2c703b4c6748 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:27.110197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f89182b5-ed91-4774-9d78-2c703b4c6748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:27.193429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 486/968] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:16:27.217732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9468a-09e4-42f9-80ae-46a9b94bf58b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:27.232092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:27.253266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868baf1-0de2-4051-8660-d64d072877a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:27.253937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868baf1-0de2-4051-8660-d64d072877a8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:27.254215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868baf1-0de2-4051-8660-d64d072877a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:27.254483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868baf1-0de2-4051-8660-d64d072877a8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:27.263108 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868baf1-0de2-4051-8660-d64d072877a8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:27.264648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868baf1-0de2-4051-8660-d64d072877a8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-4868baf1-0de2-4051-8660-d64d072877a8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:27.277359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:27.278765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:27.279716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:27.279716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:27.279716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:27.279716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:27.279716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:27.279716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:27.279716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:27.279716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:27.279716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 04:16:27.279716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:27.279716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 483/969] 10.222.0.60 () {56 vars in 1117 bytes} [Sat Sep 17 04:16:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:27.284585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89182b5-ed91-4774-9d78-2c703b4c6748 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:27.297115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:27.385223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89182b5-ed91-4774-9d78-2c703b4c6748 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:27.419008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a0566-050a-4c25-85b9-62a98e91452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:27.419622 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a0566-050a-4c25-85b9-62a98e91452b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:27.419893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a0566-050a-4c25-85b9-62a98e91452b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:27.420159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a0566-050a-4c25-85b9-62a98e91452b None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:27.420891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a0566-050a-4c25-85b9-62a98e91452b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:27.421213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a0566-050a-4c25-85b9-62a98e91452b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:27.422653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a0566-050a-4c25-85b9-62a98e91452b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:27.440117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 487/970] 10.222.0.60 () {56 vars in 1149 bytes} [Sat Sep 17 04:16:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:27.445888 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868baf1-0de2-4051-8660-d64d072877a8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:27.483942 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a0566-050a-4c25-85b9-62a98e91452b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:27.555603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:27.560979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868baf1-0de2-4051-8660-d64d072877a8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:27.598697 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a0566-050a-4c25-85b9-62a98e91452b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:27.605166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b01ea-8db5-4ac6-af08-1df9b9977db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:27.605792 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b01ea-8db5-4ac6-af08-1df9b9977db6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:27.606072 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b01ea-8db5-4ac6-af08-1df9b9977db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:27.606379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b01ea-8db5-4ac6-af08-1df9b9977db6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:27.635043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb402b3b-f2fb-4e54-a11c-90c61089f672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:27.636023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb402b3b-f2fb-4e54-a11c-90c61089f672 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:27.636612 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb402b3b-f2fb-4e54-a11c-90c61089f672 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:27.637214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb402b3b-f2fb-4e54-a11c-90c61089f672 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:27.638070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb402b3b-f2fb-4e54-a11c-90c61089f672 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:27.639812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb402b3b-f2fb-4e54-a11c-90c61089f672 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:27.655888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.sql.core [None req-cb402b3b-f2fb-4e54-a11c-90c61089f672 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 17 04:16:27.655888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 17 04:16:27.655888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [parameters: {'user_id': '6822ef2bbdc548ada4e81e3a817a1d84', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 17 04:16:27.655888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81417) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 04:16:27.657115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-cb402b3b-f2fb-4e54-a11c-90c61089f672 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application conn.query(q) Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result.read() Sep 17 04:16:27.666606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 04:16:27.667541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application next(self.gen) Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application yield resource Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application next(self.gen) Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise self.value Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application session.commit() Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 04:16:27.668674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.session.flush() Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application compat.raise_( Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 04:16:27.669634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 04:16:27.672165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application conn.query(q) Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result.read() Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 04:16:27.674269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [parameters: {'user_id': '6822ef2bbdc548ada4e81e3a817a1d84', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:27.676286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 17 04:16:27.678233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 17 04:16:27.686245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 17 04:16:27.686245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 17 04:16:27.686245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 04:16:27.686245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 04:16:27.686245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 17 04:16:27.686245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:27.687027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b01ea-8db5-4ac6-af08-1df9b9977db6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:27.687027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b01ea-8db5-4ac6-af08-1df9b9977db6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:27.687672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 488/971] 10.222.0.60 () {60 vars in 1131 bytes} [Sat Sep 17 04:16:27 2022] POST /identity/v3/users => generated 160 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:27.690148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb402b3b-f2fb-4e54-a11c-90c61089f672 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:27.789908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 484/972] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:16:27 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:16:27.790816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b01ea-8db5-4ac6-af08-1df9b9977db6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:27.797847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb402b3b-f2fb-4e54-a11c-90c61089f672 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:27.829907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:27.845108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60fedd-3ba7-4b3c-8319-e3181bed5208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:27.845108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60fedd-3ba7-4b3c-8319-e3181bed5208 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:27.845108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60fedd-3ba7-4b3c-8319-e3181bed5208 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:27.845108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60fedd-3ba7-4b3c-8319-e3181bed5208 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:27.851676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60fedd-3ba7-4b3c-8319-e3181bed5208 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:27.853562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60fedd-3ba7-4b3c-8319-e3181bed5208 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5c60fedd-3ba7-4b3c-8319-e3181bed5208 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:27.868829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:27.869839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:27.870813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:27.870813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:27.870813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:27.870813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:27.870813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:27.870813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:27.870813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:27.870813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:27.870813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 04:16:27.870813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:27.870813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 489/973] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:16:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:27.871844 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:27.878093 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60fedd-3ba7-4b3c-8319-e3181bed5208 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:27.897909 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b01ea-8db5-4ac6-af08-1df9b9977db6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:27.935197 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021db66-1e3f-44db-a06c-beb822b472eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:27.936185 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021db66-1e3f-44db-a06c-beb822b472eb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:27.936600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021db66-1e3f-44db-a06c-beb822b472eb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:27.937111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021db66-1e3f-44db-a06c-beb822b472eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:27.948193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021db66-1e3f-44db-a06c-beb822b472eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:27.950089 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021db66-1e3f-44db-a06c-beb822b472eb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-1021db66-1e3f-44db-a06c-beb822b472eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:27.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:27.967360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:27.968549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:27.968549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:27.968549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:27.970066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 485/974] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:27.975799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021db66-1e3f-44db-a06c-beb822b472eb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:27.997606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60fedd-3ba7-4b3c-8319-e3181bed5208 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:28.031789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6be74-8075-4ef8-b8aa-631ea9244c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:28.032814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6be74-8075-4ef8-b8aa-631ea9244c92 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:28.033326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6be74-8075-4ef8-b8aa-631ea9244c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:28.033787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6be74-8075-4ef8-b8aa-631ea9244c92 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:28.034681 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6be74-8075-4ef8-b8aa-631ea9244c92 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:28.035269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6be74-8075-4ef8-b8aa-631ea9244c92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:28.036994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6be74-8075-4ef8-b8aa-631ea9244c92 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:28.079198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 490/975] 10.222.0.60 () {56 vars in 1101 bytes} [Sat Sep 17 04:16:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:28.079704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021db66-1e3f-44db-a06c-beb822b472eb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:28.108976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:28.118835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123e3ab-fc69-4d1a-8f70-a11837414542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:28.120057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123e3ab-fc69-4d1a-8f70-a11837414542 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:28.120717 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123e3ab-fc69-4d1a-8f70-a11837414542 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:28.121267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123e3ab-fc69-4d1a-8f70-a11837414542 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:28.122285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123e3ab-fc69-4d1a-8f70-a11837414542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:28.122648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123e3ab-fc69-4d1a-8f70-a11837414542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:28.124100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123e3ab-fc69-4d1a-8f70-a11837414542 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:28.137648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 486/976] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:28.143694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6be74-8075-4ef8-b8aa-631ea9244c92 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:28.196741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:28.255324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6be74-8075-4ef8-b8aa-631ea9244c92 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:28.291964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b786c47-80f2-4788-95e1-8120e2dd92a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:28.292610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b786c47-80f2-4788-95e1-8120e2dd92a2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:28.292887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b786c47-80f2-4788-95e1-8120e2dd92a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:28.293185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b786c47-80f2-4788-95e1-8120e2dd92a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:28.300081 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b786c47-80f2-4788-95e1-8120e2dd92a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:28.301616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b786c47-80f2-4788-95e1-8120e2dd92a2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6b786c47-80f2-4788-95e1-8120e2dd92a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:28.314910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:28.315954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:28.317835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:28.317835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:28.317835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:28.317835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:28.317835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:28.317835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:28.317835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:28.317835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:28.317835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 491/977] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:28.325381 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123e3ab-fc69-4d1a-8f70-a11837414542 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:28.433500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123e3ab-fc69-4d1a-8f70-a11837414542 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:28.436745 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0843510-d0b6-4880-af13-8126e9c8f6ab None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:28.437026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0843510-d0b6-4880-af13-8126e9c8f6ab None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:28.437344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0843510-d0b6-4880-af13-8126e9c8f6ab None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:28.438829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 492/978] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:16:28.443717 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ccc1a-e74b-423b-8716-b17e7e59213d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:28.443992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ccc1a-e74b-423b-8716-b17e7e59213d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:28.444261 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ccc1a-e74b-423b-8716-b17e7e59213d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:28.462954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fb2ccc1a-e74b-423b-8716-b17e7e59213d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:28.464585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fb2ccc1a-e74b-423b-8716-b17e7e59213d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:28.469700 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6f622-ffec-4b11-9ce7-57edb1e9fbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:28.470314 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6f622-ffec-4b11-9ce7-57edb1e9fbb6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:28.470587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6f622-ffec-4b11-9ce7-57edb1e9fbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:28.470854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6f622-ffec-4b11-9ce7-57edb1e9fbb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:28.471540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6f622-ffec-4b11-9ce7-57edb1e9fbb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:28.471836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6f622-ffec-4b11-9ce7-57edb1e9fbb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:28.473280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6f622-ffec-4b11-9ce7-57edb1e9fbb6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:28.490432 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 487/979] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:28.491544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6f622-ffec-4b11-9ce7-57edb1e9fbb6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:28.535861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 493/980] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:16:28.543721 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:28.598503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6f622-ffec-4b11-9ce7-57edb1e9fbb6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:28.621875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b61aa4-6ef5-4759-98ef-a0c14f90d0d8 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:28.622162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b61aa4-6ef5-4759-98ef-a0c14f90d0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:28.622438 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b61aa4-6ef5-4759-98ef-a0c14f90d0d8 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:28.623940 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 494/981] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:16:28.629356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ca2b3-6578-4e95-a0c2-c386b6d175a4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:28.629656 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ca2b3-6578-4e95-a0c2-c386b6d175a4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:28.629904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ca2b3-6578-4e95-a0c2-c386b6d175a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:28.633156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23bf49-f4a3-4317-9f2b-b6c79efbed41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:28.633803 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23bf49-f4a3-4317-9f2b-b6c79efbed41 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:28.634079 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23bf49-f4a3-4317-9f2b-b6c79efbed41 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:28.634346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23bf49-f4a3-4317-9f2b-b6c79efbed41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:28.635072 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23bf49-f4a3-4317-9f2b-b6c79efbed41 None admin] RBAC: Adding query filter params (role.id=ed3c392a05fc45baac04fa3092869141, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=7c1a7d2677184c24ade81c397b7d09d1) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:28.635375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23bf49-f4a3-4317-9f2b-b6c79efbed41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:28.636786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23bf49-f4a3-4317-9f2b-b6c79efbed41 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:28.649125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 488/982] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:16:28 2022] GET /identity/v3/role_assignments?user.id=7c1a7d2677184c24ade81c397b7d09d1&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=ed3c392a05fc45baac04fa3092869141 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:28.654235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-797ca2b3-6578-4e95-a0c2-c386b6d175a4 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:28.655466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-797ca2b3-6578-4e95-a0c2-c386b6d175a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:28.691268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:28.729627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 495/983] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:28.799398 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23bf49-f4a3-4317-9f2b-b6c79efbed41 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:28.902317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23bf49-f4a3-4317-9f2b-b6c79efbed41 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:28.913281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:28.937393 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbb99d-4dcc-4b2e-b511-ae6f6ab9d5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:28.938242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbb99d-4dcc-4b2e-b511-ae6f6ab9d5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:28.938687 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbb99d-4dcc-4b2e-b511-ae6f6ab9d5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:28.939097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbb99d-4dcc-4b2e-b511-ae6f6ab9d5c5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:28.939819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fbb99d-4dcc-4b2e-b511-ae6f6ab9d5c5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:28.941411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fbb99d-4dcc-4b2e-b511-ae6f6ab9d5c5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-24fbb99d-4dcc-4b2e-b511-ae6f6ab9d5c5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:28.950322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:28.951371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:28.952351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:28.957158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 489/984] 10.222.0.60 () {56 vars in 1126 bytes} [Sat Sep 17 04:16:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:28.958591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ca2b3-6578-4e95-a0c2-c386b6d175a4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:28.996773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbb99d-4dcc-4b2e-b511-ae6f6ab9d5c5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:29.061651 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ca2b3-6578-4e95-a0c2-c386b6d175a4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:29.097632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc32a99-f773-41f0-a1a6-5bcc4df38ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:29.098239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc32a99-f773-41f0-a1a6-5bcc4df38ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:29.098509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc32a99-f773-41f0-a1a6-5bcc4df38ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:29.098777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc32a99-f773-41f0-a1a6-5bcc4df38ce4 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:29.099400 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc32a99-f773-41f0-a1a6-5bcc4df38ce4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:29.099698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc32a99-f773-41f0-a1a6-5bcc4df38ce4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:29.101069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc32a99-f773-41f0-a1a6-5bcc4df38ce4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:29.102750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbb99d-4dcc-4b2e-b511-ae6f6ab9d5c5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:29.110729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 496/985] 10.222.0.60 () {56 vars in 1122 bytes} [Sat Sep 17 04:16:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:29.111954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc32a99-f773-41f0-a1a6-5bcc4df38ce4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:29.137808 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e6684-903e-4a14-a83a-a9a4e5c8860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:29.138422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e6684-903e-4a14-a83a-a9a4e5c8860a None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:29.138696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e6684-903e-4a14-a83a-a9a4e5c8860a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:29.138964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e6684-903e-4a14-a83a-a9a4e5c8860a None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:29.139612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e6684-903e-4a14-a83a-a9a4e5c8860a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:29.140977 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e6684-903e-4a14-a83a-a9a4e5c8860a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:29.151608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:29.158856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 490/986] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:16:28 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:16:29.228317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc32a99-f773-41f0-a1a6-5bcc4df38ce4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:29.272944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286abec7-6959-4c93-8f99-0ab37bd4285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:29.273588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286abec7-6959-4c93-8f99-0ab37bd4285a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:29.273912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286abec7-6959-4c93-8f99-0ab37bd4285a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:29.274231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286abec7-6959-4c93-8f99-0ab37bd4285a None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:29.274929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286abec7-6959-4c93-8f99-0ab37bd4285a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:29.275227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286abec7-6959-4c93-8f99-0ab37bd4285a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:29.276642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286abec7-6959-4c93-8f99-0ab37bd4285a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:29.282448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 497/987] 10.222.0.60 () {56 vars in 1122 bytes} [Sat Sep 17 04:16:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:29.288415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e6684-903e-4a14-a83a-a9a4e5c8860a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:29.326645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:29.398875 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e6684-903e-4a14-a83a-a9a4e5c8860a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:29.434684 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa134e5-5470-433e-bcac-eb15d29d6ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:29.435309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa134e5-5470-433e-bcac-eb15d29d6ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:29.435586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa134e5-5470-433e-bcac-eb15d29d6ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:29.435849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa134e5-5470-433e-bcac-eb15d29d6ae2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:29.436564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa134e5-5470-433e-bcac-eb15d29d6ae2 None admin] RBAC: Adding query filter params (interface=admin, service_id=909173668f2a4a97abe351e0a0f3d5cd, region_id=RegionOne) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:29.436874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa134e5-5470-433e-bcac-eb15d29d6ae2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:29.438342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa134e5-5470-433e-bcac-eb15d29d6ae2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:29.443893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 491/988] 10.222.0.60 () {56 vars in 1249 bytes} [Sat Sep 17 04:16:29 2022] GET /identity/v3/endpoints?service_id=909173668f2a4a97abe351e0a0f3d5cd&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:29.449651 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286abec7-6959-4c93-8f99-0ab37bd4285a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:29.543425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:29.554174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286abec7-6959-4c93-8f99-0ab37bd4285a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:29.589999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d3b38-8691-4e7d-a8c1-d524cdbd6606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:29.590610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d3b38-8691-4e7d-a8c1-d524cdbd6606 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:29.590876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d3b38-8691-4e7d-a8c1-d524cdbd6606 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:29.591145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d3b38-8691-4e7d-a8c1-d524cdbd6606 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:29.591730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d3b38-8691-4e7d-a8c1-d524cdbd6606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:29.593095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d3b38-8691-4e7d-a8c1-d524cdbd6606 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:29.599484 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 498/989] 10.222.0.60 () {56 vars in 1087 bytes} [Sat Sep 17 04:16:29 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:16:30.814629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb373513-9d52-4d99-98b4-bf16c42db8cf None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:30.815186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb373513-9d52-4d99-98b4-bf16c42db8cf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:30.815186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb373513-9d52-4d99-98b4-bf16c42db8cf None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:30.816656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 492/990] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:30.821972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f1f40-59c7-41cb-a41b-7d300d90f2e9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:30.822442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f1f40-59c7-41cb-a41b-7d300d90f2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:30.822442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f1f40-59c7-41cb-a41b-7d300d90f2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:30.841345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0a2f1f40-59c7-41cb-a41b-7d300d90f2e9 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:30.842537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0a2f1f40-59c7-41cb-a41b-7d300d90f2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:30.911771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 499/991] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:16:31.168914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb373513-9d52-4d99-98b4-bf16c42db8cf None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:31.275661 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:31.281430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb373513-9d52-4d99-98b4-bf16c42db8cf None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:31.314660 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf129d7c-d903-4ebe-8c2c-89193117193b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:31.314947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf129d7c-d903-4ebe-8c2c-89193117193b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:31.315252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf129d7c-d903-4ebe-8c2c-89193117193b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:31.315866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5f172-b0d7-4c25-8feb-67e472eb9d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:31.316686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5f172-b0d7-4c25-8feb-67e472eb9d5b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:31.316792 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 500/992] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:16:31.316868 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5f172-b0d7-4c25-8feb-67e472eb9d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:31.318092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5f172-b0d7-4c25-8feb-67e472eb9d5b None admin] PATH_INFO: `/v3/services/image` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:31.318092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5f172-b0d7-4c25-8feb-67e472eb9d5b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:31.319005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5f172-b0d7-4c25-8feb-67e472eb9d5b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:31.323104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f3897-2a7b-439a-aea1-41b56ffe27b5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:31.323310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f3897-2a7b-439a-aea1-41b56ffe27b5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:31.323573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f3897-2a7b-439a-aea1-41b56ffe27b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-0ca5f172-b0d7-4c25-8feb-67e472eb9d5b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:31.328319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:31.329299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:31.330179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 493/993] 10.222.0.60 () {56 vars in 1105 bytes} [Sat Sep 17 04:16:31 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:31.335171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5f172-b0d7-4c25-8feb-67e472eb9d5b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:31.345905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-472f3897-2a7b-439a-aea1-41b56ffe27b5 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:31.347093 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-472f3897-2a7b-439a-aea1-41b56ffe27b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:31.425525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 501/994] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:31.440317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5f172-b0d7-4c25-8feb-67e472eb9d5b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:31.474616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c077851-08e0-4239-9f6e-5fbf3edb0864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:31.475224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c077851-08e0-4239-9f6e-5fbf3edb0864 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:31.475498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c077851-08e0-4239-9f6e-5fbf3edb0864 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:31.475768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c077851-08e0-4239-9f6e-5fbf3edb0864 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:31.476394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c077851-08e0-4239-9f6e-5fbf3edb0864 None admin] RBAC: Adding query filter params (name=image) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:31.476689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c077851-08e0-4239-9f6e-5fbf3edb0864 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:31.478134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c077851-08e0-4239-9f6e-5fbf3edb0864 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:31.482950 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 494/995] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:31.488751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f3897-2a7b-439a-aea1-41b56ffe27b5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:31.507274 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:31.589946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f3897-2a7b-439a-aea1-41b56ffe27b5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:31.628929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ecd62-03f0-4dc2-b9ea-4c0c371a12f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:31.629581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ecd62-03f0-4dc2-b9ea-4c0c371a12f9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:31.629857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ecd62-03f0-4dc2-b9ea-4c0c371a12f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:31.630191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ecd62-03f0-4dc2-b9ea-4c0c371a12f9 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:31.630840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ecd62-03f0-4dc2-b9ea-4c0c371a12f9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:31.631138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ecd62-03f0-4dc2-b9ea-4c0c371a12f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:31.632523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ecd62-03f0-4dc2-b9ea-4c0c371a12f9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:31.637701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 502/996] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:31 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:31.686325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c077851-08e0-4239-9f6e-5fbf3edb0864 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:31.716401 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:31.719608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc0e4c-41b5-4649-b34a-42e372f33c59 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:31.719897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc0e4c-41b5-4649-b34a-42e372f33c59 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:31.720164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc0e4c-41b5-4649-b34a-42e372f33c59 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:31.721667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 503/997] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:16:31.727453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d097ee-d6cd-447b-b13c-0e31bd5d0eb8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:31.727783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d097ee-d6cd-447b-b13c-0e31bd5d0eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:31.727990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d097ee-d6cd-447b-b13c-0e31bd5d0eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:31.746958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-58d097ee-d6cd-447b-b13c-0e31bd5d0eb8 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:31.748167 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-58d097ee-d6cd-447b-b13c-0e31bd5d0eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:31.800229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c077851-08e0-4239-9f6e-5fbf3edb0864 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:31.826840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 504/998] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:16:31.846750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc9621-ac8e-4ac3-b870-2eed2d1518eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:31.847406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc9621-ac8e-4ac3-b870-2eed2d1518eb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:31.847657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc9621-ac8e-4ac3-b870-2eed2d1518eb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:31.847923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc9621-ac8e-4ac3-b870-2eed2d1518eb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:31.848507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc9621-ac8e-4ac3-b870-2eed2d1518eb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:31.849949 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc9621-ac8e-4ac3-b870-2eed2d1518eb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-23cc9621-ac8e-4ac3-b870-2eed2d1518eb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:31.861146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:31.862187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:31.863049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 495/999] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:31.868989 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d097ee-d6cd-447b-b13c-0e31bd5d0eb8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:31.881972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:31.970420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d097ee-d6cd-447b-b13c-0e31bd5d0eb8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:32.006277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a8aa0-547c-410e-9e2c-610a42dac6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:32.006886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a8aa0-547c-410e-9e2c-610a42dac6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:32.007158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a8aa0-547c-410e-9e2c-610a42dac6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:32.007427 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a8aa0-547c-410e-9e2c-610a42dac6f7 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:32.008052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a8aa0-547c-410e-9e2c-610a42dac6f7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:32.008349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a8aa0-547c-410e-9e2c-610a42dac6f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:32.009810 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a8aa0-547c-410e-9e2c-610a42dac6f7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:32.019331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 505/1000] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:16:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:32.020872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc9621-ac8e-4ac3-b870-2eed2d1518eb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:32.087955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a8aa0-547c-410e-9e2c-610a42dac6f7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:32.117714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:32.128718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc9621-ac8e-4ac3-b870-2eed2d1518eb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:32.169119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:32.176092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c362217-8479-4ec1-aaa6-58465a82cd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:32.176712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c362217-8479-4ec1-aaa6-58465a82cd25 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:32.176960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c362217-8479-4ec1-aaa6-58465a82cd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:32.177259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c362217-8479-4ec1-aaa6-58465a82cd25 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:32.178166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c362217-8479-4ec1-aaa6-58465a82cd25 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:32.178469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c362217-8479-4ec1-aaa6-58465a82cd25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:32.179838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c362217-8479-4ec1-aaa6-58465a82cd25 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:32.185167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 496/1001] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:16:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:32.191740 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c362217-8479-4ec1-aaa6-58465a82cd25 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:32.199672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a8aa0-547c-410e-9e2c-610a42dac6f7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:32.236307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e054f-7af4-4d59-a5df-233120eef9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:32.236857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e054f-7af4-4d59-a5df-233120eef9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:32.237171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e054f-7af4-4d59-a5df-233120eef9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:32.237420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e054f-7af4-4d59-a5df-233120eef9e1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:32.237997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e054f-7af4-4d59-a5df-233120eef9e1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:32.239357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e054f-7af4-4d59-a5df-233120eef9e1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-fb8e054f-7af4-4d59-a5df-233120eef9e1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:32.249101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:32.250125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:32.251071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 506/1002] 10.222.0.60 () {56 vars in 1126 bytes} [Sat Sep 17 04:16:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:32.255987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e054f-7af4-4d59-a5df-233120eef9e1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:32.300443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c362217-8479-4ec1-aaa6-58465a82cd25 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:32.335433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45458ee5-96c7-4eb4-a3cb-6ee668f70b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:32.336149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45458ee5-96c7-4eb4-a3cb-6ee668f70b91 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:32.336419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45458ee5-96c7-4eb4-a3cb-6ee668f70b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:32.336755 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45458ee5-96c7-4eb4-a3cb-6ee668f70b91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:32.337477 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45458ee5-96c7-4eb4-a3cb-6ee668f70b91 None admin] RBAC: Adding query filter params (interface=public, service_id=e60d7e6f884e484e96a4467111abd142, region_id=RegionOne) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:32.337774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45458ee5-96c7-4eb4-a3cb-6ee668f70b91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:32.339349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45458ee5-96c7-4eb4-a3cb-6ee668f70b91 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:32.344679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 497/1003] 10.222.0.60 () {56 vars in 1251 bytes} [Sat Sep 17 04:16:32 2022] GET /identity/v3/endpoints?service_id=e60d7e6f884e484e96a4467111abd142&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:32.350640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45458ee5-96c7-4eb4-a3cb-6ee668f70b91 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:32.359798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e054f-7af4-4d59-a5df-233120eef9e1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:32.371693 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:32.395113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d6406-28ac-484e-aa3c-a14e66fa831d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:32.395732 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d6406-28ac-484e-aa3c-a14e66fa831d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:32.396002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d6406-28ac-484e-aa3c-a14e66fa831d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:32.396271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d6406-28ac-484e-aa3c-a14e66fa831d None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:32.396902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d6406-28ac-484e-aa3c-a14e66fa831d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:32.397231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d6406-28ac-484e-aa3c-a14e66fa831d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:32.398608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d6406-28ac-484e-aa3c-a14e66fa831d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:32.405121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 507/1004] 10.222.0.60 () {56 vars in 1122 bytes} [Sat Sep 17 04:16:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:32.409100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d6406-28ac-484e-aa3c-a14e66fa831d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:32.411632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:32.464423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45458ee5-96c7-4eb4-a3cb-6ee668f70b91 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:32.500556 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07ab59-ba60-419f-9e87-924d3e06547a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:32.501410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07ab59-ba60-419f-9e87-924d3e06547a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:32.501756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07ab59-ba60-419f-9e87-924d3e06547a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:32.502182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07ab59-ba60-419f-9e87-924d3e06547a None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:32.503098 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07ab59-ba60-419f-9e87-924d3e06547a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:32.504472 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07ab59-ba60-419f-9e87-924d3e06547a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:32.511163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 498/1005] 10.222.0.60 () {56 vars in 1087 bytes} [Sat Sep 17 04:16:32 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:16:32.511313 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d6406-28ac-484e-aa3c-a14e66fa831d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:32.560585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b1286-c61b-4f6b-bfca-355a1480eca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:32.561251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b1286-c61b-4f6b-bfca-355a1480eca9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:32.561526 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b1286-c61b-4f6b-bfca-355a1480eca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:32.561792 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b1286-c61b-4f6b-bfca-355a1480eca9 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:32.562440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b1286-c61b-4f6b-bfca-355a1480eca9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:32.562737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b1286-c61b-4f6b-bfca-355a1480eca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:32.564120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b1286-c61b-4f6b-bfca-355a1480eca9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:32.575208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 508/1006] 10.222.0.60 () {56 vars in 1122 bytes} [Sat Sep 17 04:16:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:32.576306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07ab59-ba60-419f-9e87-924d3e06547a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:32.673601 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:32.695237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07ab59-ba60-419f-9e87-924d3e06547a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:32.730223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2fdb4-a011-4f30-9ea1-ec4ab72a758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:32.730832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2fdb4-a011-4f30-9ea1-ec4ab72a758a None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:32.731105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2fdb4-a011-4f30-9ea1-ec4ab72a758a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:32.731367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2fdb4-a011-4f30-9ea1-ec4ab72a758a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:32.732027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2fdb4-a011-4f30-9ea1-ec4ab72a758a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:32.733427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2fdb4-a011-4f30-9ea1-ec4ab72a758a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:32.757249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 499/1007] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:16:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:16:33.813213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eacdd2-845e-4f38-b247-5f1a5bc27e52 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:33.813779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eacdd2-845e-4f38-b247-5f1a5bc27e52 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:33.813779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eacdd2-845e-4f38-b247-5f1a5bc27e52 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:33.815159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 509/1008] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:33.820482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf8a2f-b822-471b-813b-f3821a0ac816 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:33.820912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf8a2f-b822-471b-813b-f3821a0ac816 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:33.821219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf8a2f-b822-471b-813b-f3821a0ac816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:33.839545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-36cf8a2f-b822-471b-813b-f3821a0ac816 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:33.840715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-36cf8a2f-b822-471b-813b-f3821a0ac816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:33.915107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 500/1009] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:34.184689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eacdd2-845e-4f38-b247-5f1a5bc27e52 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:34.253192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:34.291654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eacdd2-845e-4f38-b247-5f1a5bc27e52 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:34.327713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3b40a-b9e9-4967-a9cc-3faa02927788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:34.328323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3b40a-b9e9-4967-a9cc-3faa02927788 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:34.328592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3b40a-b9e9-4967-a9cc-3faa02927788 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:34.328856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3b40a-b9e9-4967-a9cc-3faa02927788 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:34.329537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3b40a-b9e9-4967-a9cc-3faa02927788 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:34.330901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3b40a-b9e9-4967-a9cc-3faa02927788 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:34.347247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 510/1010] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:16:34 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:16:34.704859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be250f26-1f85-43c6-a8a1-2b393f43c7a4 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:34.705262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be250f26-1f85-43c6-a8a1-2b393f43c7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:34.705529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be250f26-1f85-43c6-a8a1-2b393f43c7a4 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:34.706954 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 501/1011] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:34.712420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a8e7c-e44d-46e0-9229-9943584c120c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:34.712683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a8e7c-e44d-46e0-9229-9943584c120c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:34.712967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a8e7c-e44d-46e0-9229-9943584c120c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:34.731850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f57a8e7c-e44d-46e0-9229-9943584c120c None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:34.733029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f57a8e7c-e44d-46e0-9229-9943584c120c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:34.808702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 511/1012] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:34 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:34.989661 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6834e87-1724-4b29-b476-caf207effd5c None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:34.990180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6834e87-1724-4b29-b476-caf207effd5c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:34.990180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6834e87-1724-4b29-b476-caf207effd5c None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:34.991953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 502/1013] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:34.997309 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d54081-92f7-40b5-b24b-07b03a9c9ae9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:34.997746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d54081-92f7-40b5-b24b-07b03a9c9ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:34.997833 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d54081-92f7-40b5-b24b-07b03a9c9ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:35.016549 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-19d54081-92f7-40b5-b24b-07b03a9c9ae9 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:35.017770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-19d54081-92f7-40b5-b24b-07b03a9c9ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:35.066269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:35.074742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6834e87-1724-4b29-b476-caf207effd5c None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:35.090463 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 512/1014] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:34 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:35.147191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:35.182790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6834e87-1724-4b29-b476-caf207effd5c None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:35.217382 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b24b5-cc83-4168-ab93-b893b2c88f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:35.218013 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b24b5-cc83-4168-ab93-b893b2c88f5e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:35.218286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b24b5-cc83-4168-ab93-b893b2c88f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:35.218558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b24b5-cc83-4168-ab93-b893b2c88f5e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:35.219147 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b24b5-cc83-4168-ab93-b893b2c88f5e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:35.220507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b24b5-cc83-4168-ab93-b893b2c88f5e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b14b24b5-cc83-4168-ab93-b893b2c88f5e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:35.229657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:35.230727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:35.234174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 503/1015] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:35.237581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d54081-92f7-40b5-b24b-07b03a9c9ae9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:35.338524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d54081-92f7-40b5-b24b-07b03a9c9ae9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:35.360211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b24b5-cc83-4168-ab93-b893b2c88f5e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:35.377667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98da2d1-1a5e-493b-874d-0f59bed9d550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:35.378808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98da2d1-1a5e-493b-874d-0f59bed9d550 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:35.379212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98da2d1-1a5e-493b-874d-0f59bed9d550 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:35.379481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98da2d1-1a5e-493b-874d-0f59bed9d550 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:35.380105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98da2d1-1a5e-493b-874d-0f59bed9d550 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:35.380404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98da2d1-1a5e-493b-874d-0f59bed9d550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:35.381792 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98da2d1-1a5e-493b-874d-0f59bed9d550 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:35.386660 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 513/1016] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:16:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:35.392278 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98da2d1-1a5e-493b-874d-0f59bed9d550 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:35.442930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:35.463588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b24b5-cc83-4168-ab93-b893b2c88f5e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:35.492350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:35.497676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b08f41-943f-4fbd-bab5-89081d900805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:35.498303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b08f41-943f-4fbd-bab5-89081d900805 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:35.498578 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b08f41-943f-4fbd-bab5-89081d900805 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:35.498843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b08f41-943f-4fbd-bab5-89081d900805 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:35.499937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b08f41-943f-4fbd-bab5-89081d900805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:35.501411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b08f41-943f-4fbd-bab5-89081d900805 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:35.502945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 504/1017] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:35.507149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98da2d1-1a5e-493b-874d-0f59bed9d550 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:35.508570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b08f41-943f-4fbd-bab5-89081d900805 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:35.543998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98888b02-7780-4da0-b16f-df920115753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:35.544613 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98888b02-7780-4da0-b16f-df920115753b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:35.544889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98888b02-7780-4da0-b16f-df920115753b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:35.545187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98888b02-7780-4da0-b16f-df920115753b None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:35.545818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98888b02-7780-4da0-b16f-df920115753b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:35.546120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98888b02-7780-4da0-b16f-df920115753b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:35.547496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98888b02-7780-4da0-b16f-df920115753b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:35.557347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 514/1018] 10.222.0.60 () {56 vars in 1112 bytes} [Sat Sep 17 04:16:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:35.558459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98888b02-7780-4da0-b16f-df920115753b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:35.613451 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b08f41-943f-4fbd-bab5-89081d900805 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:35.649394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a1b39-74d8-489f-b2b4-29e23db08f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:35.650109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a1b39-74d8-489f-b2b4-29e23db08f56 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:35.650389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a1b39-74d8-489f-b2b4-29e23db08f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:35.650767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a1b39-74d8-489f-b2b4-29e23db08f56 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:35.651418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a1b39-74d8-489f-b2b4-29e23db08f56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:35.653041 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a1b39-74d8-489f-b2b4-29e23db08f56 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:35.658956 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98888b02-7780-4da0-b16f-df920115753b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:35.673114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:35.688008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 505/1019] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:16:35 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:16:35.699273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee734de0-8d6f-4b0a-be94-2fde8b980875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:35.700267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee734de0-8d6f-4b0a-be94-2fde8b980875 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:35.700725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee734de0-8d6f-4b0a-be94-2fde8b980875 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:35.701217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee734de0-8d6f-4b0a-be94-2fde8b980875 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:35.702169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee734de0-8d6f-4b0a-be94-2fde8b980875 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:35.703867 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee734de0-8d6f-4b0a-be94-2fde8b980875 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:35.728861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 515/1020] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:16:35 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:16:36.497797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc012ebd-e995-4f50-93b7-da64721fc5fb None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:36.498353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc012ebd-e995-4f50-93b7-da64721fc5fb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:36.498353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc012ebd-e995-4f50-93b7-da64721fc5fb None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:36.499665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 506/1021] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:36.505150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf41cd-3cd7-4aea-8b9c-5e2252568eb5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:36.505634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf41cd-3cd7-4aea-8b9c-5e2252568eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:36.505862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf41cd-3cd7-4aea-8b9c-5e2252568eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:36.524632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-83cf41cd-3cd7-4aea-8b9c-5e2252568eb5 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:36.525832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-83cf41cd-3cd7-4aea-8b9c-5e2252568eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:36.599117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 516/1022] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:36.859638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc012ebd-e995-4f50-93b7-da64721fc5fb None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:36.917272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:36.969773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc012ebd-e995-4f50-93b7-da64721fc5fb None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:37.005359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e684f6-dcaa-4d63-a383-149d7e3cc8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:37.005976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e684f6-dcaa-4d63-a383-149d7e3cc8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:37.006251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e684f6-dcaa-4d63-a383-149d7e3cc8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:37.006524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e684f6-dcaa-4d63-a383-149d7e3cc8d5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:37.007097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e684f6-dcaa-4d63-a383-149d7e3cc8d5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:37.008476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e684f6-dcaa-4d63-a383-149d7e3cc8d5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-a1e684f6-dcaa-4d63-a383-149d7e3cc8d5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:37.017216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:37.018165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:37.021937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 507/1023] 10.222.0.60 () {56 vars in 1105 bytes} [Sat Sep 17 04:16:36 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:37.024969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf41cd-3cd7-4aea-8b9c-5e2252568eb5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:37.116783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:37.127619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf41cd-3cd7-4aea-8b9c-5e2252568eb5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:37.161978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3f2d6-7af2-4a3c-8342-0a458ec4d422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:37.162580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3f2d6-7af2-4a3c-8342-0a458ec4d422 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:37.162857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3f2d6-7af2-4a3c-8342-0a458ec4d422 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:37.163122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3f2d6-7af2-4a3c-8342-0a458ec4d422 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:37.163743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3f2d6-7af2-4a3c-8342-0a458ec4d422 None admin] RBAC: Adding query filter params (name=image) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:37.164036 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3f2d6-7af2-4a3c-8342-0a458ec4d422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:37.165478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3f2d6-7af2-4a3c-8342-0a458ec4d422 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:37.174851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 517/1024] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:37.176152 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e684f6-dcaa-4d63-a383-149d7e3cc8d5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:37.273525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e684f6-dcaa-4d63-a383-149d7e3cc8d5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:37.289829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:37.308618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bd212-a2e6-4da8-ab0a-1f32da75712e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:37.309261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bd212-a2e6-4da8-ab0a-1f32da75712e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:37.309536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bd212-a2e6-4da8-ab0a-1f32da75712e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:37.309806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bd212-a2e6-4da8-ab0a-1f32da75712e None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:37.310416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bd212-a2e6-4da8-ab0a-1f32da75712e None admin] RBAC: Adding query filter params (type=image) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:37.310767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bd212-a2e6-4da8-ab0a-1f32da75712e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:37.312100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bd212-a2e6-4da8-ab0a-1f32da75712e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:37.321820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 508/1025] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:37.323443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3f2d6-7af2-4a3c-8342-0a458ec4d422 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:37.429427 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3f2d6-7af2-4a3c-8342-0a458ec4d422 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:37.464751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eaf26-2a4e-41ed-aa05-227fba2716db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:37.465742 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eaf26-2a4e-41ed-aa05-227fba2716db None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:37.466240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eaf26-2a4e-41ed-aa05-227fba2716db None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:37.466701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eaf26-2a4e-41ed-aa05-227fba2716db None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:37.467588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993eaf26-2a4e-41ed-aa05-227fba2716db None admin] RBAC: Adding query filter params (interface=public, service_id=1ffe23987a3f497984f651960f5e473c, region_id=RegionOne) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:37.468106 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993eaf26-2a4e-41ed-aa05-227fba2716db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:37.469700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993eaf26-2a4e-41ed-aa05-227fba2716db None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:37.475402 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 518/1026] 10.222.0.60 () {56 vars in 1251 bytes} [Sat Sep 17 04:16:37 2022] GET /identity/v3/endpoints?service_id=1ffe23987a3f497984f651960f5e473c&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:37.481575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bd212-a2e6-4da8-ab0a-1f32da75712e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:37.585904 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bd212-a2e6-4da8-ab0a-1f32da75712e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:37.620256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2bdbb-8c15-4983-9322-34832cf5dae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:37.620866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2bdbb-8c15-4983-9322-34832cf5dae8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:37.621168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2bdbb-8c15-4983-9322-34832cf5dae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:37.621434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2bdbb-8c15-4983-9322-34832cf5dae8 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:37.622034 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2bdbb-8c15-4983-9322-34832cf5dae8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:37.623392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2bdbb-8c15-4983-9322-34832cf5dae8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:37.630416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 509/1027] 10.222.0.60 () {56 vars in 1087 bytes} [Sat Sep 17 04:16:37 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:16:37.903275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b720f-15c5-481e-a753-5e021d293ffc None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:37.903837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b720f-15c5-481e-a753-5e021d293ffc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:37.903837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b720f-15c5-481e-a753-5e021d293ffc None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:37.905412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 519/1028] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:37.911391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11dcb-9cce-45e8-b59b-2a8e255c3f7e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:37.911737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11dcb-9cce-45e8-b59b-2a8e255c3f7e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:37.911927 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11dcb-9cce-45e8-b59b-2a8e255c3f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:37.931066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-bea11dcb-9cce-45e8-b59b-2a8e255c3f7e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:37.932260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bea11dcb-9cce-45e8-b59b-2a8e255c3f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:37.972607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:38.003331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 510/1029] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:38.009251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af84538-3ae3-497e-846c-19c0b805f985 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:38.009514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af84538-3ae3-497e-846c-19c0b805f985 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:38.009788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af84538-3ae3-497e-846c-19c0b805f985 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:38.011292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 520/1030] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:38.016463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57449d72-f11a-4f51-9a41-2d47ce2daabc None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:38.016707 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57449d72-f11a-4f51-9a41-2d47ce2daabc None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:38.017009 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57449d72-f11a-4f51-9a41-2d47ce2daabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:38.036221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-57449d72-f11a-4f51-9a41-2d47ce2daabc None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:38.037547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-57449d72-f11a-4f51-9a41-2d47ce2daabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:38.111172 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 511/1031] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:38.271285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af84538-3ae3-497e-846c-19c0b805f985 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:38.295254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:38.372317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57449d72-f11a-4f51-9a41-2d47ce2daabc None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:38.380333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af84538-3ae3-497e-846c-19c0b805f985 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:38.415432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc54146-044e-444b-a94a-6d67208bebcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:38.416443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc54146-044e-444b-a94a-6d67208bebcf None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:38.416902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc54146-044e-444b-a94a-6d67208bebcf None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:38.417371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc54146-044e-444b-a94a-6d67208bebcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:38.418882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc54146-044e-444b-a94a-6d67208bebcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:38.420514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc54146-044e-444b-a94a-6d67208bebcf None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:38.422985 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 521/1032] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:38.428523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc54146-044e-444b-a94a-6d67208bebcf None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:38.487146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57449d72-f11a-4f51-9a41-2d47ce2daabc None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:38.490391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:38.527807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67d5f2-374d-4e5f-aa2b-977cc7c06900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:38.528986 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67d5f2-374d-4e5f-aa2b-977cc7c06900 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:38.529408 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67d5f2-374d-4e5f-aa2b-977cc7c06900 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:38.529679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67d5f2-374d-4e5f-aa2b-977cc7c06900 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:38.532353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:38.543058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67d5f2-374d-4e5f-aa2b-977cc7c06900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:38.545212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc54146-044e-444b-a94a-6d67208bebcf None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-5b67d5f2-374d-4e5f-aa2b-977cc7c06900 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:38.548111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:38.549033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:38.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:38.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:38.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:38.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:38.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:38.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:38.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:38.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:38.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:38.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:38.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 04:16:38.549981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:38.553420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 512/1033] 10.222.0.60 () {56 vars in 1120 bytes} [Sat Sep 17 04:16:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:38.555791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67d5f2-374d-4e5f-aa2b-977cc7c06900 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:38.580337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1d122-20bc-4e2a-9631-ac6de6d28951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:38.580950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1d122-20bc-4e2a-9631-ac6de6d28951 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:38.581258 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1d122-20bc-4e2a-9631-ac6de6d28951 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:38.581522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1d122-20bc-4e2a-9631-ac6de6d28951 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:38.582179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1d122-20bc-4e2a-9631-ac6de6d28951 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:38.583588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1d122-20bc-4e2a-9631-ac6de6d28951 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:38.608392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 522/1034] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:16:38 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:16:38.679489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67d5f2-374d-4e5f-aa2b-977cc7c06900 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:38.720265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015e42f-64dd-4938-b6f1-8b0ee8bc7383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:38.721240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015e42f-64dd-4938-b6f1-8b0ee8bc7383 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:38.721679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015e42f-64dd-4938-b6f1-8b0ee8bc7383 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:38.722099 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015e42f-64dd-4938-b6f1-8b0ee8bc7383 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:38.722921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015e42f-64dd-4938-b6f1-8b0ee8bc7383 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:38.723370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015e42f-64dd-4938-b6f1-8b0ee8bc7383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:38.724926 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015e42f-64dd-4938-b6f1-8b0ee8bc7383 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:38.734587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 513/1035] 10.222.0.60 () {56 vars in 1115 bytes} [Sat Sep 17 04:16:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:38.741296 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1d122-20bc-4e2a-9631-ac6de6d28951 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:38.846009 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1d122-20bc-4e2a-9631-ac6de6d28951 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:38.880714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f235ba7-b1b9-4e14-b1e7-494a354cbe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:38.881388 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f235ba7-b1b9-4e14-b1e7-494a354cbe33 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:38.881666 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f235ba7-b1b9-4e14-b1e7-494a354cbe33 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:38.881939 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f235ba7-b1b9-4e14-b1e7-494a354cbe33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:38.892727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f235ba7-b1b9-4e14-b1e7-494a354cbe33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:38.894240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f235ba7-b1b9-4e14-b1e7-494a354cbe33 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:38.896774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6f235ba7-b1b9-4e14-b1e7-494a354cbe33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:38.910363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:38.911788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:38.912693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:38.912693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:38.912693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:38.912693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 523/1036] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:38.917650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015e42f-64dd-4938-b6f1-8b0ee8bc7383 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:38.946096 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:39.020383 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015e42f-64dd-4938-b6f1-8b0ee8bc7383 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:39.054186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6818854-16a6-4b51-a84f-4ca51ba03eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:39.054789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6818854-16a6-4b51-a84f-4ca51ba03eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:39.055064 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6818854-16a6-4b51-a84f-4ca51ba03eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:39.055384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6818854-16a6-4b51-a84f-4ca51ba03eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:39.056020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6818854-16a6-4b51-a84f-4ca51ba03eb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:39.056322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6818854-16a6-4b51-a84f-4ca51ba03eb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:39.057715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6818854-16a6-4b51-a84f-4ca51ba03eb1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:39.069573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 514/1037] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:39.075165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f235ba7-b1b9-4e14-b1e7-494a354cbe33 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:39.175117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f235ba7-b1b9-4e14-b1e7-494a354cbe33 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:39.209514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbedd0-0a47-453b-97b0-cda815281db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:39.210285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbedd0-0a47-453b-97b0-cda815281db7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:39.210562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbedd0-0a47-453b-97b0-cda815281db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:39.210829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbedd0-0a47-453b-97b0-cda815281db7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:39.218503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbedd0-0a47-453b-97b0-cda815281db7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:39.219931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbedd0-0a47-453b-97b0-cda815281db7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-1dcbedd0-0a47-453b-97b0-cda815281db7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:39.230986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:39.232000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:39.233006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:39.233006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:39.233006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:39.233006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:39.233006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:39.233006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:39.233006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:39.233006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:39.233006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:16:39.233006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:39.236892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 524/1038] 10.222.0.60 () {56 vars in 1093 bytes} [Sat Sep 17 04:16:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:39.237929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6818854-16a6-4b51-a84f-4ca51ba03eb1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:39.324726 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:39.341997 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6818854-16a6-4b51-a84f-4ca51ba03eb1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:39.377608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf073f-8b30-4685-9dd5-6f49a5afb336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:39.378222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf073f-8b30-4685-9dd5-6f49a5afb336 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:39.378496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf073f-8b30-4685-9dd5-6f49a5afb336 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:39.378767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf073f-8b30-4685-9dd5-6f49a5afb336 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:39.379490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf073f-8b30-4685-9dd5-6f49a5afb336 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:39.379793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf073f-8b30-4685-9dd5-6f49a5afb336 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:39.381242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf073f-8b30-4685-9dd5-6f49a5afb336 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:39.398786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 515/1039] 10.222.0.60 () {56 vars in 1133 bytes} [Sat Sep 17 04:16:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:39.404686 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbedd0-0a47-453b-97b0-cda815281db7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:39.466463 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:39.511401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbedd0-0a47-453b-97b0-cda815281db7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:39.545090 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87fb45-cd0f-4645-9551-2b214fa3f8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:39.545727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87fb45-cd0f-4645-9551-2b214fa3f8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:39.546001 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87fb45-cd0f-4645-9551-2b214fa3f8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:39.546267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87fb45-cd0f-4645-9551-2b214fa3f8d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:39.616386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87fb45-cd0f-4645-9551-2b214fa3f8d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:39.617895 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87fb45-cd0f-4645-9551-2b214fa3f8d5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:39.678317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:39.731728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 525/1040] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:16:39 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:16:39.732841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf073f-8b30-4685-9dd5-6f49a5afb336 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:39.788340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3ca0b-726e-47de-9163-87eebada8e25 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:39.788580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3ca0b-726e-47de-9163-87eebada8e25 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:39.788842 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3ca0b-726e-47de-9163-87eebada8e25 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:39.790525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 526/1041] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:16:39.795364 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115a0e6-7906-4d3f-9e56-5031e572702b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:39.795618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115a0e6-7906-4d3f-9e56-5031e572702b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:39.795883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115a0e6-7906-4d3f-9e56-5031e572702b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:39.814903 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b115a0e6-7906-4d3f-9e56-5031e572702b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:39.816063 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b115a0e6-7906-4d3f-9e56-5031e572702b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:39.835752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf073f-8b30-4685-9dd5-6f49a5afb336 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:39.871832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4518d33-88b1-498f-bb95-06d017aeadce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:39.872567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4518d33-88b1-498f-bb95-06d017aeadce None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:39.872843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4518d33-88b1-498f-bb95-06d017aeadce None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:39.873251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4518d33-88b1-498f-bb95-06d017aeadce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:39.883782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4518d33-88b1-498f-bb95-06d017aeadce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:39.885211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 527/1042] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:16:39.885344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4518d33-88b1-498f-bb95-06d017aeadce None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:39.887898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d4518d33-88b1-498f-bb95-06d017aeadce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:39.901276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:39.902266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:39.903220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:39.903220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:39.903220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:39.903220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 516/1043] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:39.908177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115a0e6-7906-4d3f-9e56-5031e572702b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:39.994408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:40.010117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115a0e6-7906-4d3f-9e56-5031e572702b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:40.043889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dfbd7-ad65-4e31-84eb-7f16273ca671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:40.044627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dfbd7-ad65-4e31-84eb-7f16273ca671 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:40.044906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dfbd7-ad65-4e31-84eb-7f16273ca671 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:40.045245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dfbd7-ad65-4e31-84eb-7f16273ca671 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:40.045894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35dfbd7-ad65-4e31-84eb-7f16273ca671 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:40.046209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35dfbd7-ad65-4e31-84eb-7f16273ca671 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:40.047580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35dfbd7-ad65-4e31-84eb-7f16273ca671 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:40.063575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 528/1044] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:40.064812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4518d33-88b1-498f-bb95-06d017aeadce None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:40.145869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dfbd7-ad65-4e31-84eb-7f16273ca671 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:40.168857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4518d33-88b1-498f-bb95-06d017aeadce None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:40.206011 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237266c-dcec-4c0d-a8c9-108a2f13848e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:40.206642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237266c-dcec-4c0d-a8c9-108a2f13848e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:40.206906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237266c-dcec-4c0d-a8c9-108a2f13848e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:40.207173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237266c-dcec-4c0d-a8c9-108a2f13848e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:40.214035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237266c-dcec-4c0d-a8c9-108a2f13848e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:40.215463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237266c-dcec-4c0d-a8c9-108a2f13848e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e237266c-dcec-4c0d-a8c9-108a2f13848e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:40.227962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:40.228902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:40.229838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:40.229838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:40.229838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:40.229838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:40.229838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:40.229838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:40.229838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:40.229838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:40.229838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 517/1045] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:40.235001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237266c-dcec-4c0d-a8c9-108a2f13848e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:40.257894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dfbd7-ad65-4e31-84eb-7f16273ca671 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:40.289980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:40.295229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8f980-6670-469a-8d42-7adc70d7f478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:40.296373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8f980-6670-469a-8d42-7adc70d7f478 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:40.296866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8f980-6670-469a-8d42-7adc70d7f478 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:40.297382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8f980-6670-469a-8d42-7adc70d7f478 None admin] PATH_INFO: `/v3/services/image` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:40.298164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8f980-6670-469a-8d42-7adc70d7f478 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:40.299951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8f980-6670-469a-8d42-7adc70d7f478 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-69f8f980-6670-469a-8d42-7adc70d7f478 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:40.309312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:40.310350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:40.311197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 529/1046] 10.222.0.60 () {56 vars in 1105 bytes} [Sat Sep 17 04:16:40 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:40.317475 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8f980-6670-469a-8d42-7adc70d7f478 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:40.331600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:40.339033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237266c-dcec-4c0d-a8c9-108a2f13848e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:40.375944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6870d72-66cc-4d23-bffc-f0ee2ad440e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:40.376556 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6870d72-66cc-4d23-bffc-f0ee2ad440e1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:40.376830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6870d72-66cc-4d23-bffc-f0ee2ad440e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:40.377145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6870d72-66cc-4d23-bffc-f0ee2ad440e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:40.377833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6870d72-66cc-4d23-bffc-f0ee2ad440e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:40.378131 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6870d72-66cc-4d23-bffc-f0ee2ad440e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:40.379958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6870d72-66cc-4d23-bffc-f0ee2ad440e1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:40.392077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 518/1047] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:40.398769 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6870d72-66cc-4d23-bffc-f0ee2ad440e1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:40.435540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8f980-6670-469a-8d42-7adc70d7f478 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:40.472340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04778b-14ad-4c3f-8e4b-472eee1f00d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:40.472340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04778b-14ad-4c3f-8e4b-472eee1f00d4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:40.472691 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04778b-14ad-4c3f-8e4b-472eee1f00d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:40.473025 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04778b-14ad-4c3f-8e4b-472eee1f00d4 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:40.473560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04778b-14ad-4c3f-8e4b-472eee1f00d4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:40.473861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04778b-14ad-4c3f-8e4b-472eee1f00d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:40.475226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04778b-14ad-4c3f-8e4b-472eee1f00d4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:40.481117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 530/1048] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:40.486821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04778b-14ad-4c3f-8e4b-472eee1f00d4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:40.504136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6870d72-66cc-4d23-bffc-f0ee2ad440e1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:40.523979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:40.543285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679db45b-ef8d-47f1-95c5-924572c1f6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:40.543994 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679db45b-ef8d-47f1-95c5-924572c1f6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:40.544275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679db45b-ef8d-47f1-95c5-924572c1f6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:40.544543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679db45b-ef8d-47f1-95c5-924572c1f6b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:40.545296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679db45b-ef8d-47f1-95c5-924572c1f6b0 None admin] RBAC: Adding query filter params (role.id=ed3c392a05fc45baac04fa3092869141, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=d3272d57d5d24e5a990a6d7c86e6d4bc) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:40.545601 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679db45b-ef8d-47f1-95c5-924572c1f6b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:40.547239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679db45b-ef8d-47f1-95c5-924572c1f6b0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:40.553203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 519/1049] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:16:40 2022] GET /identity/v3/role_assignments?user.id=d3272d57d5d24e5a990a6d7c86e6d4bc&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=ed3c392a05fc45baac04fa3092869141 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:40.568802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:40.597667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04778b-14ad-4c3f-8e4b-472eee1f00d4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:40.634704 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa87e89-70e5-44cc-bb07-da72a950ff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:40.635337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa87e89-70e5-44cc-bb07-da72a950ff1d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:40.635634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa87e89-70e5-44cc-bb07-da72a950ff1d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:40.635862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa87e89-70e5-44cc-bb07-da72a950ff1d None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:40.636529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa87e89-70e5-44cc-bb07-da72a950ff1d None admin] RBAC: Adding query filter params (type=image) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:40.636799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa87e89-70e5-44cc-bb07-da72a950ff1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:40.638268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa87e89-70e5-44cc-bb07-da72a950ff1d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:40.644380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 531/1050] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:40.650482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679db45b-ef8d-47f1-95c5-924572c1f6b0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:40.766462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679db45b-ef8d-47f1-95c5-924572c1f6b0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:40.805252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb76815-0b6a-4d8d-abaa-959291c95955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:40.805778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76815-0b6a-4d8d-abaa-959291c95955 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:40.806055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76815-0b6a-4d8d-abaa-959291c95955 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:40.806322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76815-0b6a-4d8d-abaa-959291c95955 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:40.806983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb76815-0b6a-4d8d-abaa-959291c95955 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:40.808356 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb76815-0b6a-4d8d-abaa-959291c95955 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:40.830434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 520/1051] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:16:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:16:40.857371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05228d90-ff3a-447a-b34c-cfd8cdf67513 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:40.857684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05228d90-ff3a-447a-b34c-cfd8cdf67513 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:40.857921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05228d90-ff3a-447a-b34c-cfd8cdf67513 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:40.859614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 532/1052] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:40.865500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ee4a8-e208-4b9d-8920-f61e49ff6e38 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:40.866168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ee4a8-e208-4b9d-8920-f61e49ff6e38 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:40.866424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ee4a8-e208-4b9d-8920-f61e49ff6e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:40.885739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5b4ee4a8-e208-4b9d-8920-f61e49ff6e38 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:40.886960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5b4ee4a8-e208-4b9d-8920-f61e49ff6e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:40.919527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:40.971449 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 521/1053] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:41.241024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05228d90-ff3a-447a-b34c-cfd8cdf67513 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:41.345195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05228d90-ff3a-447a-b34c-cfd8cdf67513 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:41.381721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eaec6d-8c95-40f4-b7f5-33f15fcef530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:41.382353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eaec6d-8c95-40f4-b7f5-33f15fcef530 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:41.382630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eaec6d-8c95-40f4-b7f5-33f15fcef530 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:41.382900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eaec6d-8c95-40f4-b7f5-33f15fcef530 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:41.389822 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:41.394029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eaec6d-8c95-40f4-b7f5-33f15fcef530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-40eaec6d-8c95-40f4-b7f5-33f15fcef530 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:41.399492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:41.400440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:41.401321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:41.401321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:41.401321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:41.401321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:41.401321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:41.401321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:41.401321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:41.401321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:41.401321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:41.401321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:41.401321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:16:41.401321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:41.405631 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 533/1054] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:16:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:41.406694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ee4a8-e208-4b9d-8920-f61e49ff6e38 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:41.507657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ee4a8-e208-4b9d-8920-f61e49ff6e38 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:41.541555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bc7d5-35cc-466e-932c-07170591e13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:41.542167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bc7d5-35cc-466e-932c-07170591e13c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:41.542448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bc7d5-35cc-466e-932c-07170591e13c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:41.542715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bc7d5-35cc-466e-932c-07170591e13c None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:41.543468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29bc7d5-35cc-466e-932c-07170591e13c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:41.543660 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29bc7d5-35cc-466e-932c-07170591e13c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:41.545035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29bc7d5-35cc-466e-932c-07170591e13c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:41.553886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 522/1055] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:16:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:41.559860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eaec6d-8c95-40f4-b7f5-33f15fcef530 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:41.665149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eaec6d-8c95-40f4-b7f5-33f15fcef530 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:41.700490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600bdd2-24bd-4bbc-9f91-953b1b81cd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:41.701150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600bdd2-24bd-4bbc-9f91-953b1b81cd1f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:41.701444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600bdd2-24bd-4bbc-9f91-953b1b81cd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:41.701709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600bdd2-24bd-4bbc-9f91-953b1b81cd1f None admin] PATH_INFO: `/v3/users/ab0ba79278294d4b8fb32a865bf5a9fe` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:41.715240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600bdd2-24bd-4bbc-9f91-953b1b81cd1f None admin] RBAC: Authorizing `identity:get_user(user_id=ab0ba79278294d4b8fb32a865bf5a9fe)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:41.716673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600bdd2-24bd-4bbc-9f91-953b1b81cd1f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:41.723010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 534/1056] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:16:41 2022] GET /identity/v3/users/ab0ba79278294d4b8fb32a865bf5a9fe => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:41.724195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bc7d5-35cc-466e-932c-07170591e13c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:41.753686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:41.829007 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bc7d5-35cc-466e-932c-07170591e13c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:41.868766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199a175-8d38-4e2b-9256-852a22717e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:41.869419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199a175-8d38-4e2b-9256-852a22717e1c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:41.869696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199a175-8d38-4e2b-9256-852a22717e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:41.869963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199a175-8d38-4e2b-9256-852a22717e1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:41.937121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199a175-8d38-4e2b-9256-852a22717e1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:41.938654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199a175-8d38-4e2b-9256-852a22717e1c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:41.962146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:42.041187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 523/1057] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:16:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 04:16:42.043110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600bdd2-24bd-4bbc-9f91-953b1b81cd1f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:42.099195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:42.143067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600bdd2-24bd-4bbc-9f91-953b1b81cd1f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:42.176474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ddfed1-cdcc-4e9b-bb42-e3618be46594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:42.177114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddfed1-cdcc-4e9b-bb42-e3618be46594 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:42.177396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddfed1-cdcc-4e9b-bb42-e3618be46594 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:42.177663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddfed1-cdcc-4e9b-bb42-e3618be46594 None admin] PATH_INFO: `/v3/projects/03421c85728d4bb4b1484fee59189213` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:42.178791 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ddfed1-cdcc-4e9b-bb42-e3618be46594 None admin] RBAC: Authorizing `identity:get_project(project_id=03421c85728d4bb4b1484fee59189213)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:42.180275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ddfed1-cdcc-4e9b-bb42-e3618be46594 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:42.182127 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 535/1058] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:16:42 2022] GET /identity/v3/projects/03421c85728d4bb4b1484fee59189213 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:42.188134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199a175-8d38-4e2b-9256-852a22717e1c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:42.291090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199a175-8d38-4e2b-9256-852a22717e1c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:42.316442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:42.327009 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21b9a5-32eb-4feb-9b31-f03e3f1e6558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:42.327618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21b9a5-32eb-4feb-9b31-f03e3f1e6558 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:42.327943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21b9a5-32eb-4feb-9b31-f03e3f1e6558 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:42.328211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21b9a5-32eb-4feb-9b31-f03e3f1e6558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:42.328928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21b9a5-32eb-4feb-9b31-f03e3f1e6558 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=03421c85728d4bb4b1484fee59189213, user.id=ab0ba79278294d4b8fb32a865bf5a9fe) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:42.329335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21b9a5-32eb-4feb-9b31-f03e3f1e6558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:42.330761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21b9a5-32eb-4feb-9b31-f03e3f1e6558 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:42.341455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 524/1059] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:16:42 2022] GET /identity/v3/role_assignments?user.id=ab0ba79278294d4b8fb32a865bf5a9fe&scope.project.id=03421c85728d4bb4b1484fee59189213&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:42.669044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962da9a-9b1d-4ae6-91c8-30e6153fad0d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:42.669536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962da9a-9b1d-4ae6-91c8-30e6153fad0d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:42.669536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962da9a-9b1d-4ae6-91c8-30e6153fad0d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:42.671091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 536/1060] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:42.676317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868db4a3-a6b1-450c-8739-30b15dfa784a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:42.676559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868db4a3-a6b1-450c-8739-30b15dfa784a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:42.676862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868db4a3-a6b1-450c-8739-30b15dfa784a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:42.695657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-868db4a3-a6b1-450c-8739-30b15dfa784a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:42.696960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-868db4a3-a6b1-450c-8739-30b15dfa784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:42.764773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 525/1061] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:43.022784 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962da9a-9b1d-4ae6-91c8-30e6153fad0d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:43.029444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6fae9-cc77-48c4-b427-e0af23f98df2 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:43.029938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6fae9-cc77-48c4-b427-e0af23f98df2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:43.030017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6fae9-cc77-48c4-b427-e0af23f98df2 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:43.031341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 526/1062] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:43.036580 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8dcb-a699-4778-b377-73d0b6553c0e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:43.036837 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8dcb-a699-4778-b377-73d0b6553c0e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:43.037178 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8dcb-a699-4778-b377-73d0b6553c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:43.055278 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7cfc8dcb-a699-4778-b377-73d0b6553c0e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:43.056662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7cfc8dcb-a699-4778-b377-73d0b6553c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:43.122565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 527/1063] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:16:43.127336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962da9a-9b1d-4ae6-91c8-30e6153fad0d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:43.150686 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:43.162866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7475d-1c27-418d-a22c-d7abca11dcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:43.163490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7475d-1c27-418d-a22c-d7abca11dcb8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:43.163759 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7475d-1c27-418d-a22c-d7abca11dcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:43.164022 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7475d-1c27-418d-a22c-d7abca11dcb8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:43.174552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7475d-1c27-418d-a22c-d7abca11dcb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c5b7475d-1c27-418d-a22c-d7abca11dcb8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:43.179516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:43.180522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:43.181391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:43.181391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:43.181391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:43.181391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:43.181391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:43.181391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:43.181391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:43.181391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:43.181391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:43.181391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:43.181391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 04:16:43.181391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:43.185516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 537/1064] 10.222.0.60 () {56 vars in 1120 bytes} [Sat Sep 17 04:16:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:43.187202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc8dcb-a699-4778-b377-73d0b6553c0e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:43.238843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:43.291774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc8dcb-a699-4778-b377-73d0b6553c0e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:43.326664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156c64d-1832-4c5a-bb65-564448bfc6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:43.327269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156c64d-1832-4c5a-bb65-564448bfc6dc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:43.327543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156c64d-1832-4c5a-bb65-564448bfc6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:43.327811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156c64d-1832-4c5a-bb65-564448bfc6dc None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:43.328469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156c64d-1832-4c5a-bb65-564448bfc6dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:43.328768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156c64d-1832-4c5a-bb65-564448bfc6dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:43.330164 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156c64d-1832-4c5a-bb65-564448bfc6dc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:43.338451 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 528/1065] 10.222.0.60 () {56 vars in 1115 bytes} [Sat Sep 17 04:16:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:43.344045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7475d-1c27-418d-a22c-d7abca11dcb8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:43.385666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156c64d-1832-4c5a-bb65-564448bfc6dc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:43.452061 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7475d-1c27-418d-a22c-d7abca11dcb8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:43.486811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156c64d-1832-4c5a-bb65-564448bfc6dc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:43.488200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6cf01-af6e-4308-b1b3-bc8772d1a802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:43.488878 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6cf01-af6e-4308-b1b3-bc8772d1a802 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:43.489298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6cf01-af6e-4308-b1b3-bc8772d1a802 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:43.489568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6cf01-af6e-4308-b1b3-bc8772d1a802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:43.498333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:43.500261 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6cf01-af6e-4308-b1b3-bc8772d1a802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:43.502147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6cf01-af6e-4308-b1b3-bc8772d1a802 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-23e6cf01-af6e-4308-b1b3-bc8772d1a802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:43.517982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:43.519725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:43.521254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:43.521254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:43.521254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:43.523965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 538/1066] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:43.524805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6cf01-af6e-4308-b1b3-bc8772d1a802 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:43.531112 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7756816-e2fb-40b1-a9ea-a9e5da5f782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:43.531734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7756816-e2fb-40b1-a9ea-a9e5da5f782e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:43.532013 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7756816-e2fb-40b1-a9ea-a9e5da5f782e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:43.532282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7756816-e2fb-40b1-a9ea-a9e5da5f782e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:43.562002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:43.606538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7756816-e2fb-40b1-a9ea-a9e5da5f782e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:43.607987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7756816-e2fb-40b1-a9ea-a9e5da5f782e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:43.635436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6cf01-af6e-4308-b1b3-bc8772d1a802 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:43.676180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4156f-7cef-4f9b-97ca-5ca7481dc2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:43.676885 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4156f-7cef-4f9b-97ca-5ca7481dc2ca None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:43.677070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4156f-7cef-4f9b-97ca-5ca7481dc2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:43.677360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4156f-7cef-4f9b-97ca-5ca7481dc2ca None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:43.677998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4156f-7cef-4f9b-97ca-5ca7481dc2ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:43.678289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4156f-7cef-4f9b-97ca-5ca7481dc2ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:43.679651 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4156f-7cef-4f9b-97ca-5ca7481dc2ca None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:43.691778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 539/1067] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:43.697498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4156f-7cef-4f9b-97ca-5ca7481dc2ca None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:43.717170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 529/1068] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:16:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 04:16:43.722978 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7756816-e2fb-40b1-a9ea-a9e5da5f782e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:43.734155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:43.800019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:43.806813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4156f-7cef-4f9b-97ca-5ca7481dc2ca None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:43.827382 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7756816-e2fb-40b1-a9ea-a9e5da5f782e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:43.840493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be068923-a51b-4d63-b135-a6ac22ef0881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:43.841158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be068923-a51b-4d63-b135-a6ac22ef0881 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:43.841443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be068923-a51b-4d63-b135-a6ac22ef0881 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:43.841714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be068923-a51b-4d63-b135-a6ac22ef0881 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:43.848771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be068923-a51b-4d63-b135-a6ac22ef0881 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:43.850253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be068923-a51b-4d63-b135-a6ac22ef0881 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-be068923-a51b-4d63-b135-a6ac22ef0881 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:43.861884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:43.863059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:43.864245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:43.864245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:43.864245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:43.864245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:43.864245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:43.864245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:43.864245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:43.864245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:43.864245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:16:43.864245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:43.864245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 540/1069] 10.222.0.60 () {56 vars in 1093 bytes} [Sat Sep 17 04:16:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:43.867153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef9420-dbd1-4d8d-8f2f-65258610e4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:43.868373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef9420-dbd1-4d8d-8f2f-65258610e4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:43.868813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef9420-dbd1-4d8d-8f2f-65258610e4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:43.869054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be068923-a51b-4d63-b135-a6ac22ef0881 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:43.869506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef9420-dbd1-4d8d-8f2f-65258610e4e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:43.881491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef9420-dbd1-4d8d-8f2f-65258610e4e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:43.883148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef9420-dbd1-4d8d-8f2f-65258610e4e5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b0ef9420-dbd1-4d8d-8f2f-65258610e4e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:43.899311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:43.900467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:43.901664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:43.901664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:43.901664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:43.908177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 530/1070] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:43.909246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef9420-dbd1-4d8d-8f2f-65258610e4e5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:43.971897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be068923-a51b-4d63-b135-a6ac22ef0881 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:43.972490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:44.010339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e7d7e-b28e-4ef7-8729-d7bd23b8c114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:44.011285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e7d7e-b28e-4ef7-8729-d7bd23b8c114 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:44.011750 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e7d7e-b28e-4ef7-8729-d7bd23b8c114 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:44.012090 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e7d7e-b28e-4ef7-8729-d7bd23b8c114 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:44.012887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e7d7e-b28e-4ef7-8729-d7bd23b8c114 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:44.013223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e7d7e-b28e-4ef7-8729-d7bd23b8c114 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:44.014092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef9420-dbd1-4d8d-8f2f-65258610e4e5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:44.014650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e7d7e-b28e-4ef7-8729-d7bd23b8c114 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:44.041292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 541/1071] 10.222.0.60 () {56 vars in 1133 bytes} [Sat Sep 17 04:16:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:44.046942 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e7d7e-b28e-4ef7-8729-d7bd23b8c114 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:44.047920 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a486ef-8c4a-4638-aba6-fe27cac75df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:44.048548 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a486ef-8c4a-4638-aba6-fe27cac75df9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:44.048825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a486ef-8c4a-4638-aba6-fe27cac75df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:44.049103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a486ef-8c4a-4638-aba6-fe27cac75df9 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:44.049987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a486ef-8c4a-4638-aba6-fe27cac75df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:44.051140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a486ef-8c4a-4638-aba6-fe27cac75df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:44.052570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a486ef-8c4a-4638-aba6-fe27cac75df9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:44.062483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:44.076846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 531/1072] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:44.157251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e7d7e-b28e-4ef7-8729-d7bd23b8c114 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:44.194492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b59e62-4ed7-4f25-92e8-308d6e0b0be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:44.195148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b59e62-4ed7-4f25-92e8-308d6e0b0be8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:44.195416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b59e62-4ed7-4f25-92e8-308d6e0b0be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:44.195683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b59e62-4ed7-4f25-92e8-308d6e0b0be8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:44.207289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b59e62-4ed7-4f25-92e8-308d6e0b0be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:44.208764 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b59e62-4ed7-4f25-92e8-308d6e0b0be8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f3b59e62-4ed7-4f25-92e8-308d6e0b0be8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:44.224933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:44.226065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:44.226964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:44.226964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:44.226964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:44.226964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 542/1073] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:44.234136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a486ef-8c4a-4638-aba6-fe27cac75df9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:44.342325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a486ef-8c4a-4638-aba6-fe27cac75df9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:44.368899 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:44.381645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb5e8b-1941-4d83-ba24-4bd369588fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:44.382270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb5e8b-1941-4d83-ba24-4bd369588fdc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:44.382566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb5e8b-1941-4d83-ba24-4bd369588fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:44.382836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb5e8b-1941-4d83-ba24-4bd369588fdc None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:44.383533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb5e8b-1941-4d83-ba24-4bd369588fdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:44.383833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb5e8b-1941-4d83-ba24-4bd369588fdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:44.385221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb5e8b-1941-4d83-ba24-4bd369588fdc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:44.396917 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 532/1074] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:44.402609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b59e62-4ed7-4f25-92e8-308d6e0b0be8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:44.455072 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ad177-117a-4201-b2a7-43203d4435d1 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:44.455271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ad177-117a-4201-b2a7-43203d4435d1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:44.455538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ad177-117a-4201-b2a7-43203d4435d1 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:44.457387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 533/1075] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:16:44.462304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43a595-d549-4376-b403-2e4f1c595899 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:44.462582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43a595-d549-4376-b403-2e4f1c595899 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:44.462859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43a595-d549-4376-b403-2e4f1c595899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:44.468275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:44.480816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ec43a595-d549-4376-b403-2e4f1c595899 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:44.482330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ec43a595-d549-4376-b403-2e4f1c595899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:44.506654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b59e62-4ed7-4f25-92e8-308d6e0b0be8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:44.539978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2b78b-4b20-47a3-b75b-f0d0801dcd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:44.540633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2b78b-4b20-47a3-b75b-f0d0801dcd65 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:44.540898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2b78b-4b20-47a3-b75b-f0d0801dcd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:44.541191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2b78b-4b20-47a3-b75b-f0d0801dcd65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:44.547746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2b78b-4b20-47a3-b75b-f0d0801dcd65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:44.548632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 534/1076] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:16:44.549368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2b78b-4b20-47a3-b75b-f0d0801dcd65 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-98d2b78b-4b20-47a3-b75b-f0d0801dcd65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:44.560909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:44.561876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:44.562775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:44.562775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:44.562775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:44.562775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:44.562775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:44.562775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:44.562775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:44.562775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:44.562775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 543/1077] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:44.567841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43a595-d549-4376-b403-2e4f1c595899 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:44.667393 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43a595-d549-4376-b403-2e4f1c595899 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:44.681937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:44.704774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ac905-df61-45ca-85c7-a5fe181068f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:44.705473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ac905-df61-45ca-85c7-a5fe181068f5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:44.705699 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ac905-df61-45ca-85c7-a5fe181068f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:44.705966 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ac905-df61-45ca-85c7-a5fe181068f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:44.706655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ac905-df61-45ca-85c7-a5fe181068f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:44.706954 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ac905-df61-45ca-85c7-a5fe181068f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:44.708380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ac905-df61-45ca-85c7-a5fe181068f5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:44.720003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 535/1078] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:44.726193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2b78b-4b20-47a3-b75b-f0d0801dcd65 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:44.815470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ac905-df61-45ca-85c7-a5fe181068f5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:44.827080 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2b78b-4b20-47a3-b75b-f0d0801dcd65 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:44.853700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:44.863298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8dc04-56c1-4f1f-afb1-c9ac6ada7e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:44.863954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8dc04-56c1-4f1f-afb1-c9ac6ada7e3a None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:44.864230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8dc04-56c1-4f1f-afb1-c9ac6ada7e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:44.864502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8dc04-56c1-4f1f-afb1-c9ac6ada7e3a None admin] PATH_INFO: `/v3/projects/58466195aca94b188d82c1b3a647e767/users/d3272d57d5d24e5a990a6d7c86e6d4bc/roles/ed3c392a05fc45baac04fa3092869141` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:44.867100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8dc04-56c1-4f1f-afb1-c9ac6ada7e3a None admin] RBAC: Authorizing `identity:create_grant(project_id=58466195aca94b188d82c1b3a647e767, user_id=d3272d57d5d24e5a990a6d7c86e6d4bc, role_id=ed3c392a05fc45baac04fa3092869141)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:44.868536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8dc04-56c1-4f1f-afb1-c9ac6ada7e3a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:44.882167 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 544/1079] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:16:44 2022] PUT /identity/v3/projects/58466195aca94b188d82c1b3a647e767/users/d3272d57d5d24e5a990a6d7c86e6d4bc/roles/ed3c392a05fc45baac04fa3092869141 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:16:44.920209 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ac905-df61-45ca-85c7-a5fe181068f5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:44.957859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10cece-d757-444a-91c4-4b8c8cb1b493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:44.958429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10cece-d757-444a-91c4-4b8c8cb1b493 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:44.958704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10cece-d757-444a-91c4-4b8c8cb1b493 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:44.958973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10cece-d757-444a-91c4-4b8c8cb1b493 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:44.971288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10cece-d757-444a-91c4-4b8c8cb1b493 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-1d10cece-d757-444a-91c4-4b8c8cb1b493 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:44.976229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:44.977400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:44.978252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 536/1080] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:16:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:44.983921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8dc04-56c1-4f1f-afb1-c9ac6ada7e3a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:45.095307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8dc04-56c1-4f1f-afb1-c9ac6ada7e3a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:45.129922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6350e-f7ed-40d7-a2fe-501b30d53065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:45.130525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6350e-f7ed-40d7-a2fe-501b30d53065 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:45.130802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6350e-f7ed-40d7-a2fe-501b30d53065 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:45.131075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6350e-f7ed-40d7-a2fe-501b30d53065 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:45.131734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6350e-f7ed-40d7-a2fe-501b30d53065 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:45.132034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6350e-f7ed-40d7-a2fe-501b30d53065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:45.133453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6350e-f7ed-40d7-a2fe-501b30d53065 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:45.142178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 545/1081] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:16:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:45.148423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10cece-d757-444a-91c4-4b8c8cb1b493 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:45.180039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:45.246557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10cece-d757-444a-91c4-4b8c8cb1b493 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:45.280158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340adf45-6929-4e29-b456-5a57233f3527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:45.280784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340adf45-6929-4e29-b456-5a57233f3527 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:45.281061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340adf45-6929-4e29-b456-5a57233f3527 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:45.281363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340adf45-6929-4e29-b456-5a57233f3527 None admin] PATH_INFO: `/v3/users/ab0ba79278294d4b8fb32a865bf5a9fe` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:45.282528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340adf45-6929-4e29-b456-5a57233f3527 None admin] RBAC: Authorizing `identity:get_user(user_id=ab0ba79278294d4b8fb32a865bf5a9fe)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:45.283962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340adf45-6929-4e29-b456-5a57233f3527 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:45.285763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 537/1082] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:16:45 2022] GET /identity/v3/users/ab0ba79278294d4b8fb32a865bf5a9fe => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:45.291818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6350e-f7ed-40d7-a2fe-501b30d53065 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:45.341980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:45.393327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6350e-f7ed-40d7-a2fe-501b30d53065 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:45.428299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ff047-5f7a-41fc-9d78-1ca2bea84122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:45.428926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ff047-5f7a-41fc-9d78-1ca2bea84122 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:45.429218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ff047-5f7a-41fc-9d78-1ca2bea84122 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:45.429482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ff047-5f7a-41fc-9d78-1ca2bea84122 None admin] PATH_INFO: `/v3/projects/03421c85728d4bb4b1484fee59189213` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:45.430652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ff047-5f7a-41fc-9d78-1ca2bea84122 None admin] RBAC: Authorizing `identity:get_project(project_id=03421c85728d4bb4b1484fee59189213)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:45.432098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ff047-5f7a-41fc-9d78-1ca2bea84122 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:45.433714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 546/1083] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:16:45 2022] GET /identity/v3/projects/03421c85728d4bb4b1484fee59189213 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:45.440083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340adf45-6929-4e29-b456-5a57233f3527 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:45.536114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:45.539789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340adf45-6929-4e29-b456-5a57233f3527 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:45.574237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ed012-d56f-4086-a5ef-55e53acdb5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:45.574896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ed012-d56f-4086-a5ef-55e53acdb5e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:45.575168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ed012-d56f-4086-a5ef-55e53acdb5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:45.575434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ed012-d56f-4086-a5ef-55e53acdb5e6 None admin] PATH_INFO: `/v3/projects/03421c85728d4bb4b1484fee59189213/users/ab0ba79278294d4b8fb32a865bf5a9fe/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:45.577465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ed012-d56f-4086-a5ef-55e53acdb5e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=03421c85728d4bb4b1484fee59189213, user_id=ab0ba79278294d4b8fb32a865bf5a9fe, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:45.578894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ed012-d56f-4086-a5ef-55e53acdb5e6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:45.593188 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 538/1084] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:16:45 2022] PUT /identity/v3/projects/03421c85728d4bb4b1484fee59189213/users/ab0ba79278294d4b8fb32a865bf5a9fe/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:16:46.995479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fefab-c751-4aee-b240-b4391c20ea99 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:46.996032 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fefab-c751-4aee-b240-b4391c20ea99 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:46.996032 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fefab-c751-4aee-b240-b4391c20ea99 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:46.997325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 547/1085] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:47.002648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb7c65-49f3-4355-807c-081a54d03299 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:47.003078 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb7c65-49f3-4355-807c-081a54d03299 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:47.003327 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb7c65-49f3-4355-807c-081a54d03299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:47.022993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c1eb7c65-49f3-4355-807c-081a54d03299 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:47.024179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c1eb7c65-49f3-4355-807c-081a54d03299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:47.093120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 539/1086] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:47.354578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fefab-c751-4aee-b240-b4391c20ea99 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:47.456059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fefab-c751-4aee-b240-b4391c20ea99 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:47.472864 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:47.492127 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaaf3a-abf5-4f20-ab9c-a284906c9bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:47.492855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaaf3a-abf5-4f20-ab9c-a284906c9bbc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:47.493160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaaf3a-abf5-4f20-ab9c-a284906c9bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:47.493428 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaaf3a-abf5-4f20-ab9c-a284906c9bbc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:47.504726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfaaf3a-abf5-4f20-ab9c-a284906c9bbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-7dfaaf3a-abf5-4f20-ab9c-a284906c9bbc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:47.510188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:47.511382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:47.512218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:47.512218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:47.512218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:47.512218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:47.512218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:47.512218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:47.512218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:47.512218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:47.512218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:47.512218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:47.512218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 04:16:47.512218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:47.512866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 548/1087] 10.222.0.60 () {56 vars in 1120 bytes} [Sat Sep 17 04:16:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:47.519354 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb7c65-49f3-4355-807c-081a54d03299 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:47.613352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:47.626207 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb7c65-49f3-4355-807c-081a54d03299 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:47.663328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e28024-67d3-4b9e-a169-098fdc10fd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:47.663945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e28024-67d3-4b9e-a169-098fdc10fd46 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:47.664223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e28024-67d3-4b9e-a169-098fdc10fd46 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:47.664600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e28024-67d3-4b9e-a169-098fdc10fd46 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:47.665334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e28024-67d3-4b9e-a169-098fdc10fd46 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:47.665640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e28024-67d3-4b9e-a169-098fdc10fd46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:47.667021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e28024-67d3-4b9e-a169-098fdc10fd46 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:47.676416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 540/1088] 10.222.0.60 () {56 vars in 1115 bytes} [Sat Sep 17 04:16:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:47.682456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaaf3a-abf5-4f20-ab9c-a284906c9bbc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:47.727385 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8055d-7eb8-4a7b-aa00-77adf7b8232c None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:47.727664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8055d-7eb8-4a7b-aa00-77adf7b8232c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:47.727938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8055d-7eb8-4a7b-aa00-77adf7b8232c None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:47.729307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 541/1089] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:47.734514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f740d29-d914-4192-a6f0-d06d4cf768c0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:47.734790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f740d29-d914-4192-a6f0-d06d4cf768c0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:47.735066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f740d29-d914-4192-a6f0-d06d4cf768c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:47.754693 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-8f740d29-d914-4192-a6f0-d06d4cf768c0 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:47.755871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8f740d29-d914-4192-a6f0-d06d4cf768c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:47.785370 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaaf3a-abf5-4f20-ab9c-a284906c9bbc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:47.818513 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7558d-9e13-415b-a356-c06f6a6d1c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:47.819142 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7558d-9e13-415b-a356-c06f6a6d1c5c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:47.819415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7558d-9e13-415b-a356-c06f6a6d1c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:47.819679 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7558d-9e13-415b-a356-c06f6a6d1c5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:47.820652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 542/1090] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:16:47.829735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7558d-9e13-415b-a356-c06f6a6d1c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:47.831189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7558d-9e13-415b-a356-c06f6a6d1c5c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d5d7558d-9e13-415b-a356-c06f6a6d1c5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:47.845572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:47.846593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:47.847503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:47.847503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:47.847503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:47.847503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 549/1091] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:47.852473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f740d29-d914-4192-a6f0-d06d4cf768c0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:47.936854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:47.958959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f740d29-d914-4192-a6f0-d06d4cf768c0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:47.994151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea7eac-682e-4a38-aba6-ee4c48e2cf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:47.994757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea7eac-682e-4a38-aba6-ee4c48e2cf80 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:47.995032 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea7eac-682e-4a38-aba6-ee4c48e2cf80 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:47.995275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea7eac-682e-4a38-aba6-ee4c48e2cf80 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:47.995911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea7eac-682e-4a38-aba6-ee4c48e2cf80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:47.996211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea7eac-682e-4a38-aba6-ee4c48e2cf80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:47.997617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea7eac-682e-4a38-aba6-ee4c48e2cf80 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:48.008929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 543/1092] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:48.014873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7558d-9e13-415b-a356-c06f6a6d1c5c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:48.043888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:48.117110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7558d-9e13-415b-a356-c06f6a6d1c5c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:48.133723 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea7eac-682e-4a38-aba6-ee4c48e2cf80 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:48.150868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc86fb1-9e10-4986-a24b-5beb09619b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:48.151494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc86fb1-9e10-4986-a24b-5beb09619b5a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:48.151766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc86fb1-9e10-4986-a24b-5beb09619b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:48.152029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc86fb1-9e10-4986-a24b-5beb09619b5a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:48.159306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc86fb1-9e10-4986-a24b-5beb09619b5a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:48.160721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc86fb1-9e10-4986-a24b-5beb09619b5a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-dbc86fb1-9e10-4986-a24b-5beb09619b5a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:16:48.171568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:48.172554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:48.173568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:48.173568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:48.173568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:16:48.173568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:16:48.173568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:16:48.173568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:16:48.173568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:16:48.173568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:16:48.173568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 04:16:48.173568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:48.173568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 550/1093] 10.222.0.60 () {56 vars in 1093 bytes} [Sat Sep 17 04:16:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:48.178441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc86fb1-9e10-4986-a24b-5beb09619b5a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:48.237291 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea7eac-682e-4a38-aba6-ee4c48e2cf80 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:48.263071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:48.270274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356c7d4-64cd-4d5e-8cca-91b492cda03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:48.270910 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c7d4-64cd-4d5e-8cca-91b492cda03a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:48.271186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c7d4-64cd-4d5e-8cca-91b492cda03a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:48.271454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c7d4-64cd-4d5e-8cca-91b492cda03a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:48.275617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc86fb1-9e10-4986-a24b-5beb09619b5a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:48.282423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356c7d4-64cd-4d5e-8cca-91b492cda03a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b356c7d4-64cd-4d5e-8cca-91b492cda03a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:16:48.287153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:48.288330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:16:48.289633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 544/1094] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:16:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:48.294199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356c7d4-64cd-4d5e-8cca-91b492cda03a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:48.296839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:48.316521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86772e1-7179-49f1-b829-6c567d4bc098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:48.317204 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86772e1-7179-49f1-b829-6c567d4bc098 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:48.317479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86772e1-7179-49f1-b829-6c567d4bc098 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:48.317746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86772e1-7179-49f1-b829-6c567d4bc098 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:48.318485 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86772e1-7179-49f1-b829-6c567d4bc098 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:48.318782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86772e1-7179-49f1-b829-6c567d4bc098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:48.320518 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86772e1-7179-49f1-b829-6c567d4bc098 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:48.339437 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 551/1095] 10.222.0.60 () {56 vars in 1133 bytes} [Sat Sep 17 04:16:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:48.345791 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86772e1-7179-49f1-b829-6c567d4bc098 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:48.414765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356c7d4-64cd-4d5e-8cca-91b492cda03a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:48.447630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1792c-cd25-47e8-a9ed-4a25c6d72c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:48.448243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1792c-cd25-47e8-a9ed-4a25c6d72c3e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:48.448620 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1792c-cd25-47e8-a9ed-4a25c6d72c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:48.448794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1792c-cd25-47e8-a9ed-4a25c6d72c3e None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:48.449495 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1792c-cd25-47e8-a9ed-4a25c6d72c3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:48.449819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1792c-cd25-47e8-a9ed-4a25c6d72c3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:48.451189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1792c-cd25-47e8-a9ed-4a25c6d72c3e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:48.452028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86772e1-7179-49f1-b829-6c567d4bc098 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:48.465835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 545/1096] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:16:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:48.466923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1792c-cd25-47e8-a9ed-4a25c6d72c3e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:48.485201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9327a99-f127-4812-9b97-2d149659c9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:48.485811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9327a99-f127-4812-9b97-2d149659c9ed None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:48.486086 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9327a99-f127-4812-9b97-2d149659c9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:48.486356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9327a99-f127-4812-9b97-2d149659c9ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:48.501599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:48.526413 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:48.555309 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9327a99-f127-4812-9b97-2d149659c9ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:48.556768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9327a99-f127-4812-9b97-2d149659c9ed None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:48.564373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1792c-cd25-47e8-a9ed-4a25c6d72c3e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:48.597177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc292d-cae3-4dc7-8026-328724b4bc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:48.597805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc292d-cae3-4dc7-8026-328724b4bc56 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:48.598083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc292d-cae3-4dc7-8026-328724b4bc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:48.598363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc292d-cae3-4dc7-8026-328724b4bc56 None admin] PATH_INFO: `/v3/users/ab0ba79278294d4b8fb32a865bf5a9fe` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:48.599492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc292d-cae3-4dc7-8026-328724b4bc56 None admin] RBAC: Authorizing `identity:get_user(user_id=ab0ba79278294d4b8fb32a865bf5a9fe)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:48.600914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc292d-cae3-4dc7-8026-328724b4bc56 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:48.607365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 546/1097] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:16:48 2022] GET /identity/v3/users/ab0ba79278294d4b8fb32a865bf5a9fe => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:48.608671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc292d-cae3-4dc7-8026-328724b4bc56 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:48.653024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 552/1098] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:16:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 04:16:48.658903 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9327a99-f127-4812-9b97-2d149659c9ed None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:48.703365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:48.714998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc292d-cae3-4dc7-8026-328724b4bc56 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:48.744382 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:48.750383 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd21863-09b7-4481-bf04-a62fe59decdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:48.750996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd21863-09b7-4481-bf04-a62fe59decdc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:48.751270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd21863-09b7-4481-bf04-a62fe59decdc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:48.751536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd21863-09b7-4481-bf04-a62fe59decdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:48.756780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9327a99-f127-4812-9b97-2d149659c9ed None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:48.791420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d09133-3c9a-4fdf-be93-24cf2d8468da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:48.792050 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d09133-3c9a-4fdf-be93-24cf2d8468da None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:48.792324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d09133-3c9a-4fdf-be93-24cf2d8468da None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:48.792595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d09133-3c9a-4fdf-be93-24cf2d8468da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:48.802331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d09133-3c9a-4fdf-be93-24cf2d8468da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:48.803787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d09133-3c9a-4fdf-be93-24cf2d8468da None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:48.818017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd21863-09b7-4481-bf04-a62fe59decdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-49d09133-3c9a-4fdf-be93-24cf2d8468da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:16:48.818183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:48.819350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:16:48.820760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:16:48.820760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:16:48.820760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:48.820760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 553/1099] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:16:48.821047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd21863-09b7-4481-bf04-a62fe59decdc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:48.824946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d09133-3c9a-4fdf-be93-24cf2d8468da None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:48.916386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 547/1100] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:16:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 04:16:48.917575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:48.921659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d09133-3c9a-4fdf-be93-24cf2d8468da None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:48.922474 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd21863-09b7-4481-bf04-a62fe59decdc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:48.951457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:48.955851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ab493-f126-42b3-a6d6-c94a0f05211c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:48.956528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ab493-f126-42b3-a6d6-c94a0f05211c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:48.956806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ab493-f126-42b3-a6d6-c94a0f05211c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:48.957074 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ab493-f126-42b3-a6d6-c94a0f05211c None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:48.957745 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ab493-f126-42b3-a6d6-c94a0f05211c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:48.958038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ab493-f126-42b3-a6d6-c94a0f05211c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:48.959410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ab493-f126-42b3-a6d6-c94a0f05211c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:48.970995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 554/1101] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:48.976614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ab493-f126-42b3-a6d6-c94a0f05211c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:49.019897 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd21863-09b7-4481-bf04-a62fe59decdc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:49.053092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800a2f4-79e3-4327-ad0c-a73543f34729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:49.053740 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800a2f4-79e3-4327-ad0c-a73543f34729 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:49.054017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800a2f4-79e3-4327-ad0c-a73543f34729 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:49.054288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800a2f4-79e3-4327-ad0c-a73543f34729 None admin] PATH_INFO: `/v3/projects/03421c85728d4bb4b1484fee59189213` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:49.055481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800a2f4-79e3-4327-ad0c-a73543f34729 None admin] RBAC: Authorizing `identity:get_project(project_id=03421c85728d4bb4b1484fee59189213)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:49.056921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800a2f4-79e3-4327-ad0c-a73543f34729 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:49.058747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 548/1102] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:16:48 2022] GET /identity/v3/projects/03421c85728d4bb4b1484fee59189213 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:49.064576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800a2f4-79e3-4327-ad0c-a73543f34729 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:49.071459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ab493-f126-42b3-a6d6-c94a0f05211c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:49.105143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac0931-5495-4193-88bd-ab794f5ab4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:49.106077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac0931-5495-4193-88bd-ab794f5ab4ee None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:49.106357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac0931-5495-4193-88bd-ab794f5ab4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:49.106763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac0931-5495-4193-88bd-ab794f5ab4ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:49.113273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac0931-5495-4193-88bd-ab794f5ab4ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:49.114705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac0931-5495-4193-88bd-ab794f5ab4ee None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-eaac0931-5495-4193-88bd-ab794f5ab4ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:16:49.125836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:16:49.126944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:16:49.127848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:16:49.127848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:16:49.127848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:16:49.127848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:16:49.127848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:16:49.127848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:16:49.127848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:16:49.127848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:16:49.127848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 555/1103] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:16:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:16:49.132704 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac0931-5495-4193-88bd-ab794f5ab4ee None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:49.150738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:49.159922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:49.162846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800a2f4-79e3-4327-ad0c-a73543f34729 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:49.197597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08452dba-8d51-4e5c-ad7e-d7bd194dc919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:49.198187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08452dba-8d51-4e5c-ad7e-d7bd194dc919 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:49.198473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08452dba-8d51-4e5c-ad7e-d7bd194dc919 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:49.198736 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08452dba-8d51-4e5c-ad7e-d7bd194dc919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:49.199455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08452dba-8d51-4e5c-ad7e-d7bd194dc919 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=03421c85728d4bb4b1484fee59189213, user.id=ab0ba79278294d4b8fb32a865bf5a9fe) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:49.199761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08452dba-8d51-4e5c-ad7e-d7bd194dc919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:49.201204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08452dba-8d51-4e5c-ad7e-d7bd194dc919 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:49.206895 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 549/1104] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:16:49 2022] GET /identity/v3/role_assignments?user.id=ab0ba79278294d4b8fb32a865bf5a9fe&scope.project.id=03421c85728d4bb4b1484fee59189213&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:49.230234 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac0931-5495-4193-88bd-ab794f5ab4ee None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:49.267894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7b9f8-8f93-4ca4-9b52-16f4e30f8918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:49.268516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7b9f8-8f93-4ca4-9b52-16f4e30f8918 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:49.268786 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7b9f8-8f93-4ca4-9b52-16f4e30f8918 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:49.269048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7b9f8-8f93-4ca4-9b52-16f4e30f8918 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:49.269779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7b9f8-8f93-4ca4-9b52-16f4e30f8918 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:49.270146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7b9f8-8f93-4ca4-9b52-16f4e30f8918 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:49.271558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7b9f8-8f93-4ca4-9b52-16f4e30f8918 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:49.284803 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 556/1105] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:16:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:16:49.291609 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08452dba-8d51-4e5c-ad7e-d7bd194dc919 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:49.398783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08452dba-8d51-4e5c-ad7e-d7bd194dc919 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:49.432316 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f41c42-edef-410f-b540-1f97b07f42f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:49.432934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f41c42-edef-410f-b540-1f97b07f42f7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:49.433236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f41c42-edef-410f-b540-1f97b07f42f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:49.433506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f41c42-edef-410f-b540-1f97b07f42f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:49.434218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f41c42-edef-410f-b540-1f97b07f42f7 None admin] RBAC: Adding query filter params (role.id=ed3c392a05fc45baac04fa3092869141, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=d3272d57d5d24e5a990a6d7c86e6d4bc) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:49.434523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f41c42-edef-410f-b540-1f97b07f42f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:49.435944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f41c42-edef-410f-b540-1f97b07f42f7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:49.442019 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 550/1106] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:16:49 2022] GET /identity/v3/role_assignments?user.id=d3272d57d5d24e5a990a6d7c86e6d4bc&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=ed3c392a05fc45baac04fa3092869141 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:51.377883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dd43b-cefc-4a63-bc7d-e30d030d5270 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:51.378789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dd43b-cefc-4a63-bc7d-e30d030d5270 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:51.378789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dd43b-cefc-4a63-bc7d-e30d030d5270 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:51.380894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 557/1107] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:51.387246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1115b-c7ea-4dd2-8b32-29efa891c35d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:51.387996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1115b-c7ea-4dd2-8b32-29efa891c35d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:51.388124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1115b-c7ea-4dd2-8b32-29efa891c35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:51.411358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b3f1115b-c7ea-4dd2-8b32-29efa891c35d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:51.412527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b3f1115b-c7ea-4dd2-8b32-29efa891c35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:51.436844 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:51.486456 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 551/1108] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:51.757233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dd43b-cefc-4a63-bc7d-e30d030d5270 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:51.852614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:51.882139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dd43b-cefc-4a63-bc7d-e30d030d5270 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:51.916542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df1224-db2b-4b27-adc3-cb5f4a21de7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:51.917213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df1224-db2b-4b27-adc3-cb5f4a21de7d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:51.917498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df1224-db2b-4b27-adc3-cb5f4a21de7d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:51.917766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df1224-db2b-4b27-adc3-cb5f4a21de7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:51.919064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df1224-db2b-4b27-adc3-cb5f4a21de7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:51.920439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df1224-db2b-4b27-adc3-cb5f4a21de7d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:51.922028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 558/1109] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:16:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:51.931212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1115b-c7ea-4dd2-8b32-29efa891c35d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:52.058744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1115b-c7ea-4dd2-8b32-29efa891c35d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:52.102349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28abff-0683-42ce-864c-0ed66431df79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:52.102950 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28abff-0683-42ce-864c-0ed66431df79 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:52.103215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28abff-0683-42ce-864c-0ed66431df79 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:52.103478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28abff-0683-42ce-864c-0ed66431df79 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:52.104117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28abff-0683-42ce-864c-0ed66431df79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:52.105573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28abff-0683-42ce-864c-0ed66431df79 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:52.129853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 552/1110] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:16:51 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:16:54.279819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5cb5e-4673-4590-833e-cc5660fa9c3a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:54.280383 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5cb5e-4673-4590-833e-cc5660fa9c3a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:54.280383 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5cb5e-4673-4590-833e-cc5660fa9c3a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:54.282529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 559/1111] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:54.291416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f4d05-4bc7-49a3-9fb2-b71c228f7e0c None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:54.292162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f4d05-4bc7-49a3-9fb2-b71c228f7e0c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:54.292296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f4d05-4bc7-49a3-9fb2-b71c228f7e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:54.316005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-3f2f4d05-4bc7-49a3-9fb2-b71c228f7e0c None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:54.317316 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3f2f4d05-4bc7-49a3-9fb2-b71c228f7e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:54.328524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:54.390180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 553/1112] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:54.656121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5cb5e-4673-4590-833e-cc5660fa9c3a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:54.763795 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5cb5e-4673-4590-833e-cc5660fa9c3a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:54.798196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78a8c9-e42c-46b4-8b1f-56b8d929cd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:54.798816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78a8c9-e42c-46b4-8b1f-56b8d929cd14 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:54.799090 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78a8c9-e42c-46b4-8b1f-56b8d929cd14 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:54.799358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78a8c9-e42c-46b4-8b1f-56b8d929cd14 None admin] PATH_INFO: `/v3/roles/84212031716c4cf8a5083824fe47dbb7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:54.800779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78a8c9-e42c-46b4-8b1f-56b8d929cd14 None admin] RBAC: Authorizing `identity:get_role(role_id=84212031716c4cf8a5083824fe47dbb7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:54.802186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78a8c9-e42c-46b4-8b1f-56b8d929cd14 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:54.803490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 560/1113] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:16:54 2022] GET /identity/v3/roles/84212031716c4cf8a5083824fe47dbb7 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:54.810142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f4d05-4bc7-49a3-9fb2-b71c228f7e0c None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:54.916675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f4d05-4bc7-49a3-9fb2-b71c228f7e0c None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:54.951856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671027c-ac07-4fa9-9928-29677593eb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:54.952475 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671027c-ac07-4fa9-9928-29677593eb99 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:54.952750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671027c-ac07-4fa9-9928-29677593eb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:54.953017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671027c-ac07-4fa9-9928-29677593eb99 None admin] PATH_INFO: `/v3/users/9adb6dea0655405483ef1c4467d8bb22` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:54.966175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671027c-ac07-4fa9-9928-29677593eb99 None admin] RBAC: Authorizing `identity:get_user(user_id=9adb6dea0655405483ef1c4467d8bb22)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:54.967610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671027c-ac07-4fa9-9928-29677593eb99 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:54.969194 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 554/1114] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:16:54 2022] GET /identity/v3/users/9adb6dea0655405483ef1c4467d8bb22 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:54.975494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78a8c9-e42c-46b4-8b1f-56b8d929cd14 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:54.998012 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:55.082103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78a8c9-e42c-46b4-8b1f-56b8d929cd14 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:55.121475 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaf3bf-dead-4d6d-9d0f-5915590e1329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:55.122081 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaf3bf-dead-4d6d-9d0f-5915590e1329 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:55.122354 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaf3bf-dead-4d6d-9d0f-5915590e1329 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:55.122616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaf3bf-dead-4d6d-9d0f-5915590e1329 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:55.195276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcaf3bf-dead-4d6d-9d0f-5915590e1329 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:55.196710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcaf3bf-dead-4d6d-9d0f-5915590e1329 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:55.210885 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:55.295912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 561/1115] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:16:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:16:55.305518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671027c-ac07-4fa9-9928-29677593eb99 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:55.308537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:55.414768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671027c-ac07-4fa9-9928-29677593eb99 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:55.452404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa963c1e-4d3a-4259-b0df-9b4cdc4d7b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:55.452951 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa963c1e-4d3a-4259-b0df-9b4cdc4d7b52 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:55.453257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa963c1e-4d3a-4259-b0df-9b4cdc4d7b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:55.453524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa963c1e-4d3a-4259-b0df-9b4cdc4d7b52 None admin] PATH_INFO: `/v3/projects/03421c85728d4bb4b1484fee59189213` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:55.454699 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa963c1e-4d3a-4259-b0df-9b4cdc4d7b52 None admin] RBAC: Authorizing `identity:get_project(project_id=03421c85728d4bb4b1484fee59189213)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:55.456132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa963c1e-4d3a-4259-b0df-9b4cdc4d7b52 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:55.457757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 555/1116] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:16:55 2022] GET /identity/v3/projects/03421c85728d4bb4b1484fee59189213 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:55.467181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaf3bf-dead-4d6d-9d0f-5915590e1329 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:55.571943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaf3bf-dead-4d6d-9d0f-5915590e1329 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:55.586793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:55.610767 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c28208-e821-4af9-a604-1f9728baa74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:55.611351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c28208-e821-4af9-a604-1f9728baa74c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:55.611621 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c28208-e821-4af9-a604-1f9728baa74c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:55.611880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c28208-e821-4af9-a604-1f9728baa74c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:55.612585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c28208-e821-4af9-a604-1f9728baa74c None admin] RBAC: Adding query filter params (role.id=84212031716c4cf8a5083824fe47dbb7, scope.project.id=03421c85728d4bb4b1484fee59189213, user.id=9adb6dea0655405483ef1c4467d8bb22) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:16:55.612884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c28208-e821-4af9-a604-1f9728baa74c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:55.614329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c28208-e821-4af9-a604-1f9728baa74c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:55.620293 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 562/1117] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:16:55 2022] GET /identity/v3/role_assignments?user.id=9adb6dea0655405483ef1c4467d8bb22&scope.project.id=03421c85728d4bb4b1484fee59189213&role.id=84212031716c4cf8a5083824fe47dbb7 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:57.741714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93f59e-de16-40b2-b991-16e805dcbc31 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:57.742605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93f59e-de16-40b2-b991-16e805dcbc31 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:57.742605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93f59e-de16-40b2-b991-16e805dcbc31 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:57.744743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 556/1118] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:16:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:16:57.751063 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c26c4-0b60-4623-88ff-a6c1cf5eef7d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:57.751752 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c26c4-0b60-4623-88ff-a6c1cf5eef7d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:57.751953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c26c4-0b60-4623-88ff-a6c1cf5eef7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:57.774679 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2a1c26c4-0b60-4623-88ff-a6c1cf5eef7d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:16:57.775865 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2a1c26c4-0b60-4623-88ff-a6c1cf5eef7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:16:57.846601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 563/1119] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:16:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:16:58.123405 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93f59e-de16-40b2-b991-16e805dcbc31 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:58.207773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:58.241020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93f59e-de16-40b2-b991-16e805dcbc31 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:58.277152 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdaafe-58b6-4311-a85c-1dc54b5c66f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:58.277786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdaafe-58b6-4311-a85c-1dc54b5c66f3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:58.278052 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdaafe-58b6-4311-a85c-1dc54b5c66f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:58.278309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdaafe-58b6-4311-a85c-1dc54b5c66f3 None admin] PATH_INFO: `/v3/roles/84212031716c4cf8a5083824fe47dbb7` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:58.279805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccdaafe-58b6-4311-a85c-1dc54b5c66f3 None admin] RBAC: Authorizing `identity:get_role(role_id=84212031716c4cf8a5083824fe47dbb7)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:58.281196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccdaafe-58b6-4311-a85c-1dc54b5c66f3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:58.282482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 557/1120] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:16:58 2022] GET /identity/v3/roles/84212031716c4cf8a5083824fe47dbb7 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:58.291747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c26c4-0b60-4623-88ff-a6c1cf5eef7d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:58.403499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c26c4-0b60-4623-88ff-a6c1cf5eef7d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:58.412445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:16:58.439194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7f491-5448-45ab-aa18-eaf842b8864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:58.439812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7f491-5448-45ab-aa18-eaf842b8864a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:58.440068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7f491-5448-45ab-aa18-eaf842b8864a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:58.440347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7f491-5448-45ab-aa18-eaf842b8864a None admin] PATH_INFO: `/v3/users/9adb6dea0655405483ef1c4467d8bb22` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:58.441614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7f491-5448-45ab-aa18-eaf842b8864a None admin] RBAC: Authorizing `identity:get_user(user_id=9adb6dea0655405483ef1c4467d8bb22)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:58.443044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7f491-5448-45ab-aa18-eaf842b8864a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:58.444580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 564/1121] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:16:58 2022] GET /identity/v3/users/9adb6dea0655405483ef1c4467d8bb22 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:58.450648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdaafe-58b6-4311-a85c-1dc54b5c66f3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:58.563659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdaafe-58b6-4311-a85c-1dc54b5c66f3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:58.598196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da340dd-c3d1-48fd-af91-37f4e50c7780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:58.598805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da340dd-c3d1-48fd-af91-37f4e50c7780 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:58.599077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da340dd-c3d1-48fd-af91-37f4e50c7780 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:58.599351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da340dd-c3d1-48fd-af91-37f4e50c7780 None admin] PATH_INFO: `/v3/projects/03421c85728d4bb4b1484fee59189213` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:58.600515 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da340dd-c3d1-48fd-af91-37f4e50c7780 None admin] RBAC: Authorizing `identity:get_project(project_id=03421c85728d4bb4b1484fee59189213)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:58.602012 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da340dd-c3d1-48fd-af91-37f4e50c7780 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:58.603629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 558/1122] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:16:58 2022] GET /identity/v3/projects/03421c85728d4bb4b1484fee59189213 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:16:58.610009 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7f491-5448-45ab-aa18-eaf842b8864a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:16:58.710919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7f491-5448-45ab-aa18-eaf842b8864a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:16:58.746379 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e397e7-cf93-4065-841a-670c4126a849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:16:58.747023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e397e7-cf93-4065-841a-670c4126a849 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:16:58.747300 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e397e7-cf93-4065-841a-670c4126a849 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:16:58.747570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e397e7-cf93-4065-841a-670c4126a849 None admin] PATH_INFO: `/v3/projects/03421c85728d4bb4b1484fee59189213/users/9adb6dea0655405483ef1c4467d8bb22/roles/84212031716c4cf8a5083824fe47dbb7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:16:58.750302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e397e7-cf93-4065-841a-670c4126a849 None admin] RBAC: Authorizing `identity:create_grant(project_id=03421c85728d4bb4b1484fee59189213, user_id=9adb6dea0655405483ef1c4467d8bb22, role_id=84212031716c4cf8a5083824fe47dbb7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:16:58.751737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e397e7-cf93-4065-841a-670c4126a849 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:16:58.772123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 565/1123] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:16:58 2022] PUT /identity/v3/projects/03421c85728d4bb4b1484fee59189213/users/9adb6dea0655405483ef1c4467d8bb22/roles/84212031716c4cf8a5083824fe47dbb7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:17:00.864692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81a93b-115c-45c9-a24d-35421b22e914 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:00.866010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81a93b-115c-45c9-a24d-35421b22e914 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:00.866010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81a93b-115c-45c9-a24d-35421b22e914 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:00.868017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 559/1124] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:00.873687 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aea467-b548-46b0-b83a-0c71b7e07751 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:00.874130 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aea467-b548-46b0-b83a-0c71b7e07751 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:00.874240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aea467-b548-46b0-b83a-0c71b7e07751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:00.892748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-05aea467-b548-46b0-b83a-0c71b7e07751 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:00.893978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-05aea467-b548-46b0-b83a-0c71b7e07751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:00.924477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:00.968779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 566/1125] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:17:01.243553 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81a93b-115c-45c9-a24d-35421b22e914 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:01.257707 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:01.363331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81a93b-115c-45c9-a24d-35421b22e914 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:01.398051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2438d-0755-4359-9438-2974a0c1e95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:01.398641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2438d-0755-4359-9438-2974a0c1e95e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:01.398912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2438d-0755-4359-9438-2974a0c1e95e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:01.399175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2438d-0755-4359-9438-2974a0c1e95e None admin] PATH_INFO: `/v3/roles/84212031716c4cf8a5083824fe47dbb7` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:01.400585 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2438d-0755-4359-9438-2974a0c1e95e None admin] RBAC: Authorizing `identity:get_role(role_id=84212031716c4cf8a5083824fe47dbb7)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:01.401966 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2438d-0755-4359-9438-2974a0c1e95e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:01.403265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 560/1126] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:17:01 2022] GET /identity/v3/roles/84212031716c4cf8a5083824fe47dbb7 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:01.411824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aea467-b548-46b0-b83a-0c71b7e07751 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:01.521113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aea467-b548-46b0-b83a-0c71b7e07751 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:01.555900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d8ec2-987e-4ffc-9df8-08878481a258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:01.556518 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d8ec2-987e-4ffc-9df8-08878481a258 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:01.556791 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d8ec2-987e-4ffc-9df8-08878481a258 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:01.557058 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d8ec2-987e-4ffc-9df8-08878481a258 None admin] PATH_INFO: `/v3/users/9adb6dea0655405483ef1c4467d8bb22` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:01.558231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d8ec2-987e-4ffc-9df8-08878481a258 None admin] RBAC: Authorizing `identity:get_user(user_id=9adb6dea0655405483ef1c4467d8bb22)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:01.559655 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d8ec2-987e-4ffc-9df8-08878481a258 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:01.561237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 567/1127] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:17:01 2022] GET /identity/v3/users/9adb6dea0655405483ef1c4467d8bb22 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:17:01.567063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2438d-0755-4359-9438-2974a0c1e95e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:01.652282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:01.678324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2438d-0755-4359-9438-2974a0c1e95e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:01.714709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c9977-c3d4-4ad7-ae0d-ca72c445ee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:01.715571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c9977-c3d4-4ad7-ae0d-ca72c445ee77 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:01.715931 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c9977-c3d4-4ad7-ae0d-ca72c445ee77 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:01.716246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c9977-c3d4-4ad7-ae0d-ca72c445ee77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:01.800207 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c9977-c3d4-4ad7-ae0d-ca72c445ee77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:01.801602 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c9977-c3d4-4ad7-ae0d-ca72c445ee77 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:01.881155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:01.906917 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 561/1128] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:17:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:17:01.913160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d8ec2-987e-4ffc-9df8-08878481a258 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:01.956221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:02.025919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d8ec2-987e-4ffc-9df8-08878481a258 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:02.062292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95829fd6-26c3-47fb-85e0-dc2507d3296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:02.062860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95829fd6-26c3-47fb-85e0-dc2507d3296f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:02.063133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95829fd6-26c3-47fb-85e0-dc2507d3296f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:02.063400 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95829fd6-26c3-47fb-85e0-dc2507d3296f None admin] PATH_INFO: `/v3/projects/03421c85728d4bb4b1484fee59189213` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:02.064739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95829fd6-26c3-47fb-85e0-dc2507d3296f None admin] RBAC: Authorizing `identity:get_project(project_id=03421c85728d4bb4b1484fee59189213)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:02.066243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95829fd6-26c3-47fb-85e0-dc2507d3296f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:02.067850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 568/1129] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:17:01 2022] GET /identity/v3/projects/03421c85728d4bb4b1484fee59189213 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:02.074469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c9977-c3d4-4ad7-ae0d-ca72c445ee77 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:02.205200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c9977-c3d4-4ad7-ae0d-ca72c445ee77 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:02.241075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6a5bc-73a1-46f5-8f6d-5903efbaf45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:02.241728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6a5bc-73a1-46f5-8f6d-5903efbaf45c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:02.241976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6a5bc-73a1-46f5-8f6d-5903efbaf45c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:02.242268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6a5bc-73a1-46f5-8f6d-5903efbaf45c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:02.242985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6a5bc-73a1-46f5-8f6d-5903efbaf45c None admin] RBAC: Adding query filter params (role.id=84212031716c4cf8a5083824fe47dbb7, scope.project.id=03421c85728d4bb4b1484fee59189213, user.id=9adb6dea0655405483ef1c4467d8bb22) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:17:02.243286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6a5bc-73a1-46f5-8f6d-5903efbaf45c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:02.244715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6a5bc-73a1-46f5-8f6d-5903efbaf45c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:02.250622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 562/1130] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:17:02 2022] GET /identity/v3/role_assignments?user.id=9adb6dea0655405483ef1c4467d8bb22&scope.project.id=03421c85728d4bb4b1484fee59189213&role.id=84212031716c4cf8a5083824fe47dbb7 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:04.385338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84eef12-dd50-440e-aaed-625733dc332a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:04.385910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84eef12-dd50-440e-aaed-625733dc332a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:04.385910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84eef12-dd50-440e-aaed-625733dc332a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:04.387249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 569/1131] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:04.393690 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44385626-6c4a-45b6-a1be-6f30a917824e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:04.394386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44385626-6c4a-45b6-a1be-6f30a917824e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:04.394531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44385626-6c4a-45b6-a1be-6f30a917824e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:04.422187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-44385626-6c4a-45b6-a1be-6f30a917824e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:04.423361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-44385626-6c4a-45b6-a1be-6f30a917824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:04.454148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:04.498114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 563/1132] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:17:04.759732 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84eef12-dd50-440e-aaed-625733dc332a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:04.864069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84eef12-dd50-440e-aaed-625733dc332a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:04.866002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:04.902787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620e587-2a40-44be-89f5-18fc2fdcf23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:04.903399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620e587-2a40-44be-89f5-18fc2fdcf23c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:04.903672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620e587-2a40-44be-89f5-18fc2fdcf23c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:04.903943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620e587-2a40-44be-89f5-18fc2fdcf23c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:04.905122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620e587-2a40-44be-89f5-18fc2fdcf23c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:04.906553 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620e587-2a40-44be-89f5-18fc2fdcf23c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:04.908114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 570/1133] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:17:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:04.914183 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44385626-6c4a-45b6-a1be-6f30a917824e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:05.011750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44385626-6c4a-45b6-a1be-6f30a917824e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:05.046469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a1021-35bc-4e53-9820-1e4c1bb4fe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:05.047062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a1021-35bc-4e53-9820-1e4c1bb4fe35 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:05.047340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a1021-35bc-4e53-9820-1e4c1bb4fe35 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:05.047609 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a1021-35bc-4e53-9820-1e4c1bb4fe35 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:05.048252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a1021-35bc-4e53-9820-1e4c1bb4fe35 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:05.049718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a1021-35bc-4e53-9820-1e4c1bb4fe35 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:05.081987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 564/1134] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:17:04 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:17:07.236677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdbb3b-705a-46f8-b9bf-60d1def36f50 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:07.237359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdbb3b-705a-46f8-b9bf-60d1def36f50 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:07.237359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdbb3b-705a-46f8-b9bf-60d1def36f50 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:07.238592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 571/1135] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:07.246037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c610bc-7b46-423c-976b-0f3ea169481b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:07.246766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c610bc-7b46-423c-976b-0f3ea169481b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:07.246913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c610bc-7b46-423c-976b-0f3ea169481b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:07.277641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-19c610bc-7b46-423c-976b-0f3ea169481b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:07.279537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-19c610bc-7b46-423c-976b-0f3ea169481b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:07.299079 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:07.360043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 565/1136] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:17:07.657615 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdbb3b-705a-46f8-b9bf-60d1def36f50 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:07.770048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdbb3b-705a-46f8-b9bf-60d1def36f50 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:07.808943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490c47c-1c61-4d3a-87d3-c46a00baba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:07.809596 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490c47c-1c61-4d3a-87d3-c46a00baba7b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:07.809871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490c47c-1c61-4d3a-87d3-c46a00baba7b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:07.810137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490c47c-1c61-4d3a-87d3-c46a00baba7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:07.811301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490c47c-1c61-4d3a-87d3-c46a00baba7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:07.812731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490c47c-1c61-4d3a-87d3-c46a00baba7b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:07.814369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 572/1137] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:17:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:07.823065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c610bc-7b46-423c-976b-0f3ea169481b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:07.923370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c610bc-7b46-423c-976b-0f3ea169481b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:07.961448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56021641-d498-4ba6-8457-4c2f6dff2cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:07.962044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56021641-d498-4ba6-8457-4c2f6dff2cc1 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:07.962316 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56021641-d498-4ba6-8457-4c2f6dff2cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:07.962583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56021641-d498-4ba6-8457-4c2f6dff2cc1 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:07.963225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56021641-d498-4ba6-8457-4c2f6dff2cc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:07.964650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56021641-d498-4ba6-8457-4c2f6dff2cc1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:07.987934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 566/1138] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:17:07 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:17:10.151766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5c8e0-de3d-44ef-a1af-3ae5beabddb3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:10.152669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5c8e0-de3d-44ef-a1af-3ae5beabddb3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:10.152669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5c8e0-de3d-44ef-a1af-3ae5beabddb3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:10.154852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 573/1139] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:10.161154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86235cb7-aa99-4c2f-8ff6-b99a386f08a3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:10.161845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86235cb7-aa99-4c2f-8ff6-b99a386f08a3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:10.162115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86235cb7-aa99-4c2f-8ff6-b99a386f08a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:10.176526 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:10.182511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-86235cb7-aa99-4c2f-8ff6-b99a386f08a3 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:10.184431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-86235cb7-aa99-4c2f-8ff6-b99a386f08a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:10.255411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 567/1140] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:17:10.522505 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5c8e0-de3d-44ef-a1af-3ae5beabddb3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:10.573864 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:10.644119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5c8e0-de3d-44ef-a1af-3ae5beabddb3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:10.681811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab8f4b-fc6c-4738-97f4-f5ade5d4deb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:10.682439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab8f4b-fc6c-4738-97f4-f5ade5d4deb8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:10.682710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab8f4b-fc6c-4738-97f4-f5ade5d4deb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:10.682982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab8f4b-fc6c-4738-97f4-f5ade5d4deb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:10.693529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab8f4b-fc6c-4738-97f4-f5ade5d4deb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a5ab8f4b-fc6c-4738-97f4-f5ade5d4deb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:17:10.698310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:17:10.699424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:17:10.700367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 574/1141] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:17:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:17:10.706435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86235cb7-aa99-4c2f-8ff6-b99a386f08a3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:10.826451 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86235cb7-aa99-4c2f-8ff6-b99a386f08a3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:10.868695 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4da63-50c3-4ab9-b5d3-65fb1acd28fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:10.869324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4da63-50c3-4ab9-b5d3-65fb1acd28fa None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:10.869600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4da63-50c3-4ab9-b5d3-65fb1acd28fa None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:10.869863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4da63-50c3-4ab9-b5d3-65fb1acd28fa None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:10.870525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4da63-50c3-4ab9-b5d3-65fb1acd28fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:17:10.870822 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4da63-50c3-4ab9-b5d3-65fb1acd28fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:10.872192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4da63-50c3-4ab9-b5d3-65fb1acd28fa None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:10.874405 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:10.881068 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 568/1142] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:17:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:10.889979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab8f4b-fc6c-4738-97f4-f5ade5d4deb8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:11.002287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:11.006883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab8f4b-fc6c-4738-97f4-f5ade5d4deb8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:11.042490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a37b40-b5d6-45bd-8ac6-cc6db74a4def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:11.043115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a37b40-b5d6-45bd-8ac6-cc6db74a4def None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:11.043382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a37b40-b5d6-45bd-8ac6-cc6db74a4def None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:11.043643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a37b40-b5d6-45bd-8ac6-cc6db74a4def None admin] PATH_INFO: `/v3/users/1707790692ce4156a6755b088063426f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:11.057625 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a37b40-b5d6-45bd-8ac6-cc6db74a4def None admin] RBAC: Authorizing `identity:get_user(user_id=1707790692ce4156a6755b088063426f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:11.059062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a37b40-b5d6-45bd-8ac6-cc6db74a4def None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:11.060617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 575/1143] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:17:10 2022] GET /identity/v3/users/1707790692ce4156a6755b088063426f => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:11.066688 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4da63-50c3-4ab9-b5d3-65fb1acd28fa None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:11.174618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4da63-50c3-4ab9-b5d3-65fb1acd28fa None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:11.208886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474cfe0-d60b-47d8-895b-b5c2057db2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:11.209509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474cfe0-d60b-47d8-895b-b5c2057db2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:11.209787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474cfe0-d60b-47d8-895b-b5c2057db2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:11.210054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474cfe0-d60b-47d8-895b-b5c2057db2f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:11.274939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:11.279040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474cfe0-d60b-47d8-895b-b5c2057db2f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:11.280547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474cfe0-d60b-47d8-895b-b5c2057db2f3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:11.387533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 569/1144] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:17:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:17:11.393877 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a37b40-b5d6-45bd-8ac6-cc6db74a4def None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:11.504868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a37b40-b5d6-45bd-8ac6-cc6db74a4def None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:11.539858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a781f8-d10f-4ffc-8f83-0668d13a14ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:11.540474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a781f8-d10f-4ffc-8f83-0668d13a14ce None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:11.540743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a781f8-d10f-4ffc-8f83-0668d13a14ce None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:11.541014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a781f8-d10f-4ffc-8f83-0668d13a14ce None admin] PATH_INFO: `/v3/projects/98ef67cfa23046758441cbfff27f7c87` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:11.542355 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a781f8-d10f-4ffc-8f83-0668d13a14ce None admin] RBAC: Authorizing `identity:get_project(project_id=98ef67cfa23046758441cbfff27f7c87)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:11.543781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a781f8-d10f-4ffc-8f83-0668d13a14ce None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:11.545435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 576/1145] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:17:11 2022] GET /identity/v3/projects/98ef67cfa23046758441cbfff27f7c87 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:11.551749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474cfe0-d60b-47d8-895b-b5c2057db2f3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:11.656718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:11.660721 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474cfe0-d60b-47d8-895b-b5c2057db2f3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:11.696617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51d10a-75b1-4a29-9cc7-f67da5e42f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:11.697243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51d10a-75b1-4a29-9cc7-f67da5e42f17 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:11.697517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51d10a-75b1-4a29-9cc7-f67da5e42f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:11.697775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51d10a-75b1-4a29-9cc7-f67da5e42f17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:11.698496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51d10a-75b1-4a29-9cc7-f67da5e42f17 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=98ef67cfa23046758441cbfff27f7c87, user.id=1707790692ce4156a6755b088063426f) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:17:11.698804 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51d10a-75b1-4a29-9cc7-f67da5e42f17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:11.700211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51d10a-75b1-4a29-9cc7-f67da5e42f17 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:11.707192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 570/1146] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:17:11 2022] GET /identity/v3/role_assignments?user.id=1707790692ce4156a6755b088063426f&scope.project.id=98ef67cfa23046758441cbfff27f7c87&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:13.802705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82084fbc-b2e3-4160-84a1-a69fec6d110a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:13.803604 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82084fbc-b2e3-4160-84a1-a69fec6d110a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:13.803604 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82084fbc-b2e3-4160-84a1-a69fec6d110a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:13.805536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 577/1147] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:13.811743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50f92-9930-433b-87d8-f4fc4ae93729 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:13.812430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50f92-9930-433b-87d8-f4fc4ae93729 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:13.812538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50f92-9930-433b-87d8-f4fc4ae93729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:13.840316 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5fe50f92-9930-433b-87d8-f4fc4ae93729 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:13.841566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5fe50f92-9930-433b-87d8-f4fc4ae93729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:13.914104 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 571/1148] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:17:14.189981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82084fbc-b2e3-4160-84a1-a69fec6d110a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:14.221942 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:14.306409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82084fbc-b2e3-4160-84a1-a69fec6d110a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:14.340969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c384a3-3db7-4498-95f7-3b6239e49212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:14.341620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c384a3-3db7-4498-95f7-3b6239e49212 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:14.341894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c384a3-3db7-4498-95f7-3b6239e49212 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:14.342156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c384a3-3db7-4498-95f7-3b6239e49212 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:14.352442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c384a3-3db7-4498-95f7-3b6239e49212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d3c384a3-3db7-4498-95f7-3b6239e49212 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:17:14.357211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:17:14.358415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:17:14.359462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 578/1149] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:17:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:17:14.367088 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50f92-9930-433b-87d8-f4fc4ae93729 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:14.458151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:14.467673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50f92-9930-433b-87d8-f4fc4ae93729 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:14.505742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8f81d-9d27-4061-a4a4-ae61255c7c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:14.506344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8f81d-9d27-4061-a4a4-ae61255c7c77 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:14.506617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8f81d-9d27-4061-a4a4-ae61255c7c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:14.506882 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8f81d-9d27-4061-a4a4-ae61255c7c77 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:14.507540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8f81d-9d27-4061-a4a4-ae61255c7c77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:17:14.507857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8f81d-9d27-4061-a4a4-ae61255c7c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:14.509258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8f81d-9d27-4061-a4a4-ae61255c7c77 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:14.517569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 572/1150] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:17:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:14.523742 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c384a3-3db7-4498-95f7-3b6239e49212 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:14.609008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:14.627642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c384a3-3db7-4498-95f7-3b6239e49212 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:14.663339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb5be9-bdf0-43ea-9279-681650f2003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:14.664005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb5be9-bdf0-43ea-9279-681650f2003e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:14.664258 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb5be9-bdf0-43ea-9279-681650f2003e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:14.664555 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb5be9-bdf0-43ea-9279-681650f2003e None admin] PATH_INFO: `/v3/users/1707790692ce4156a6755b088063426f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:14.665716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb5be9-bdf0-43ea-9279-681650f2003e None admin] RBAC: Authorizing `identity:get_user(user_id=1707790692ce4156a6755b088063426f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:14.667150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb5be9-bdf0-43ea-9279-681650f2003e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:14.669103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 579/1151] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:17:14 2022] GET /identity/v3/users/1707790692ce4156a6755b088063426f => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:14.677511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8f81d-9d27-4061-a4a4-ae61255c7c77 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:14.781189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8f81d-9d27-4061-a4a4-ae61255c7c77 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:14.818737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfd0d9-9c8a-4241-ad87-cc699764b75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:14.819305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfd0d9-9c8a-4241-ad87-cc699764b75b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:14.819616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfd0d9-9c8a-4241-ad87-cc699764b75b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:14.819882 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfd0d9-9c8a-4241-ad87-cc699764b75b None admin] PATH_INFO: `/v3/projects/98ef67cfa23046758441cbfff27f7c87` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:14.821139 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfd0d9-9c8a-4241-ad87-cc699764b75b None admin] RBAC: Authorizing `identity:get_project(project_id=98ef67cfa23046758441cbfff27f7c87)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:14.822569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfd0d9-9c8a-4241-ad87-cc699764b75b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:14.824195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 573/1152] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:17:14 2022] GET /identity/v3/projects/98ef67cfa23046758441cbfff27f7c87 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:14.833427 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb5be9-bdf0-43ea-9279-681650f2003e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:14.933757 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb5be9-bdf0-43ea-9279-681650f2003e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:14.968190 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d1f51-1803-4833-bb39-98e095080a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:14.968824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d1f51-1803-4833-bb39-98e095080a82 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:14.969120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d1f51-1803-4833-bb39-98e095080a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:14.969393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d1f51-1803-4833-bb39-98e095080a82 None admin] PATH_INFO: `/v3/projects/98ef67cfa23046758441cbfff27f7c87/users/1707790692ce4156a6755b088063426f/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:14.971377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d1f51-1803-4833-bb39-98e095080a82 None admin] RBAC: Authorizing `identity:create_grant(project_id=98ef67cfa23046758441cbfff27f7c87, user_id=1707790692ce4156a6755b088063426f, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:14.972790 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d1f51-1803-4833-bb39-98e095080a82 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:14.989318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 580/1153] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:17:14 2022] PUT /identity/v3/projects/98ef67cfa23046758441cbfff27f7c87/users/1707790692ce4156a6755b088063426f/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:17:17.070784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b1546-a60c-42fe-940b-fec29cdff6b0 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:17.071675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b1546-a60c-42fe-940b-fec29cdff6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:17.071675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b1546-a60c-42fe-940b-fec29cdff6b0 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:17.073872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 574/1154] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:17.079377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebdeda-b1ad-43a1-967b-851b4b9c9136 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:17.079824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebdeda-b1ad-43a1-967b-851b4b9c9136 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:17.079927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebdeda-b1ad-43a1-967b-851b4b9c9136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:17.100367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-51ebdeda-b1ad-43a1-967b-851b4b9c9136 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:17.101636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-51ebdeda-b1ad-43a1-967b-851b4b9c9136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:17.108731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:17.168387 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 581/1155] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:17:17.434805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b1546-a60c-42fe-940b-fec29cdff6b0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:17.463893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:17.544366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b1546-a60c-42fe-940b-fec29cdff6b0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:17.578911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55d1b3-7d71-4903-ad0d-486757593dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:17.579526 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55d1b3-7d71-4903-ad0d-486757593dfb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:17.579799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55d1b3-7d71-4903-ad0d-486757593dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:17.580069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55d1b3-7d71-4903-ad0d-486757593dfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:17.591549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55d1b3-7d71-4903-ad0d-486757593dfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-0d55d1b3-7d71-4903-ad0d-486757593dfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:17:17.596298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:17:17.597410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:17:17.598412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:17:17.598412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:17:17.598412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:17:17.598412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:17:17.598412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:17:17.598412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:17:17.598412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:17:17.598412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:17:17.598412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:17:17.598412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:17:17.598412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:17:17.598412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:17:17.601246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 575/1156] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:17:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:17:17.603340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebdeda-b1ad-43a1-967b-851b4b9c9136 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:17.708141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebdeda-b1ad-43a1-967b-851b4b9c9136 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:17.743118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7046c-e554-4f5c-931e-f28f804420d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:17.743713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7046c-e554-4f5c-931e-f28f804420d6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:17.743986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7046c-e554-4f5c-931e-f28f804420d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:17.744254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7046c-e554-4f5c-931e-f28f804420d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:17.744912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7046c-e554-4f5c-931e-f28f804420d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:17:17.745240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7046c-e554-4f5c-931e-f28f804420d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:17.746602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7046c-e554-4f5c-931e-f28f804420d6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:17.754650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 582/1157] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:17:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:17.761452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55d1b3-7d71-4903-ad0d-486757593dfb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:17.860758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:17.880757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55d1b3-7d71-4903-ad0d-486757593dfb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:17.915133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0e9e2-e536-47c9-b0cb-8aff0aa9b8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:17.915760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0e9e2-e536-47c9-b0cb-8aff0aa9b8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:17.916037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0e9e2-e536-47c9-b0cb-8aff0aa9b8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:17.916307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0e9e2-e536-47c9-b0cb-8aff0aa9b8a7 None admin] PATH_INFO: `/v3/users/1707790692ce4156a6755b088063426f` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:17.917514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0e9e2-e536-47c9-b0cb-8aff0aa9b8a7 None admin] RBAC: Authorizing `identity:get_user(user_id=1707790692ce4156a6755b088063426f)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:17.918972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0e9e2-e536-47c9-b0cb-8aff0aa9b8a7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:17.920527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 576/1158] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:17:17 2022] GET /identity/v3/users/1707790692ce4156a6755b088063426f => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:17.930804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7046c-e554-4f5c-931e-f28f804420d6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:17.937103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:18.037748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7046c-e554-4f5c-931e-f28f804420d6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:18.073534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da2084-f7c2-4097-a125-23104410616a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:18.074126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da2084-f7c2-4097-a125-23104410616a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:18.074402 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da2084-f7c2-4097-a125-23104410616a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:18.074676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da2084-f7c2-4097-a125-23104410616a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:18.159529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da2084-f7c2-4097-a125-23104410616a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:18.160992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da2084-f7c2-4097-a125-23104410616a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:18.163767 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:18.273127 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 583/1159] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:17:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:17:18.280087 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0e9e2-e536-47c9-b0cb-8aff0aa9b8a7 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:18.389888 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0e9e2-e536-47c9-b0cb-8aff0aa9b8a7 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:18.424096 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271310f5-46b8-4706-9b9b-6ba9b1c465c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:18.424705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271310f5-46b8-4706-9b9b-6ba9b1c465c1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:18.424975 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271310f5-46b8-4706-9b9b-6ba9b1c465c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:18.425276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271310f5-46b8-4706-9b9b-6ba9b1c465c1 None admin] PATH_INFO: `/v3/projects/98ef67cfa23046758441cbfff27f7c87` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:18.426504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271310f5-46b8-4706-9b9b-6ba9b1c465c1 None admin] RBAC: Authorizing `identity:get_project(project_id=98ef67cfa23046758441cbfff27f7c87)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:18.427932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271310f5-46b8-4706-9b9b-6ba9b1c465c1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:18.429712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 577/1160] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:17:18 2022] GET /identity/v3/projects/98ef67cfa23046758441cbfff27f7c87 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:18.438952 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da2084-f7c2-4097-a125-23104410616a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:18.542679 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da2084-f7c2-4097-a125-23104410616a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:18.545058 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:18.586815 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89e315-4835-4266-8b77-65a54c68a189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:18.587451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89e315-4835-4266-8b77-65a54c68a189 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:18.587722 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89e315-4835-4266-8b77-65a54c68a189 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:18.587995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89e315-4835-4266-8b77-65a54c68a189 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:18.588719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89e315-4835-4266-8b77-65a54c68a189 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=98ef67cfa23046758441cbfff27f7c87, user.id=1707790692ce4156a6755b088063426f) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:17:18.589028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89e315-4835-4266-8b77-65a54c68a189 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:18.590489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89e315-4835-4266-8b77-65a54c68a189 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:18.596291 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 584/1161] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:17:18 2022] GET /identity/v3/role_assignments?user.id=1707790692ce4156a6755b088063426f&scope.project.id=98ef67cfa23046758441cbfff27f7c87&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:17:20.741042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3078e62-f384-4b72-83ea-b1fcf4bb04e5 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:20.741935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3078e62-f384-4b72-83ea-b1fcf4bb04e5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:20.741935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3078e62-f384-4b72-83ea-b1fcf4bb04e5 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:20.744072 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 578/1162] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:20.749591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ca216-98a7-4923-a592-91f8941db48a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:20.750021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ca216-98a7-4923-a592-91f8941db48a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:20.750129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ca216-98a7-4923-a592-91f8941db48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:20.769920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7f3ca216-98a7-4923-a592-91f8941db48a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:20.771101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7f3ca216-98a7-4923-a592-91f8941db48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:20.841443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 585/1163] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:17:21.117302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3078e62-f384-4b72-83ea-b1fcf4bb04e5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:21.150444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:21.257102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3078e62-f384-4b72-83ea-b1fcf4bb04e5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:21.295245 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194de841-2782-450d-bdce-76bbd1649aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:21.295850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194de841-2782-450d-bdce-76bbd1649aae None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:21.296124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194de841-2782-450d-bdce-76bbd1649aae None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:21.296393 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194de841-2782-450d-bdce-76bbd1649aae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:21.366717 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194de841-2782-450d-bdce-76bbd1649aae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:21.368285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194de841-2782-450d-bdce-76bbd1649aae None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:21.385936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:21.471453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 579/1164] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:17:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:17:21.479753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ca216-98a7-4923-a592-91f8941db48a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:21.590261 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:21.597790 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ca216-98a7-4923-a592-91f8941db48a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:21.634182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2ca26-6825-4cf4-9c75-873d40d808a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:21.634732 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2ca26-6825-4cf4-9c75-873d40d808a1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:21.635003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2ca26-6825-4cf4-9c75-873d40d808a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:21.635269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2ca26-6825-4cf4-9c75-873d40d808a1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:21.642203 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2ca26-6825-4cf4-9c75-873d40d808a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:21.643643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2ca26-6825-4cf4-9c75-873d40d808a1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-21f2ca26-6825-4cf4-9c75-873d40d808a1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:17:21.655194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:17:21.656320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:17:21.657430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 586/1165] 10.222.0.60 () {56 vars in 1117 bytes} [Sat Sep 17 04:17:21 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:17:21.664782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194de841-2782-450d-bdce-76bbd1649aae None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:21.767135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194de841-2782-450d-bdce-76bbd1649aae None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:21.785921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:21.805856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01d107-d571-4c47-95f1-218d100d382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:21.806453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01d107-d571-4c47-95f1-218d100d382a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:21.806727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01d107-d571-4c47-95f1-218d100d382a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:21.806991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01d107-d571-4c47-95f1-218d100d382a None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:21.807631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01d107-d571-4c47-95f1-218d100d382a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:17:21.807927 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01d107-d571-4c47-95f1-218d100d382a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:21.809336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01d107-d571-4c47-95f1-218d100d382a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:21.816734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 580/1166] 10.222.0.60 () {56 vars in 1115 bytes} [Sat Sep 17 04:17:21 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:17:23.931192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a2b87-5371-49d1-9329-655d048717f0 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:23.931728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a2b87-5371-49d1-9329-655d048717f0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:23.931728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a2b87-5371-49d1-9329-655d048717f0 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:23.933716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 587/1167] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:23.942021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61bdb8-89db-4d8f-91ae-fcbc3449c823 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:23.942489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61bdb8-89db-4d8f-91ae-fcbc3449c823 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:23.942594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61bdb8-89db-4d8f-91ae-fcbc3449c823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:23.962503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9d61bdb8-89db-4d8f-91ae-fcbc3449c823 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:23.963676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9d61bdb8-89db-4d8f-91ae-fcbc3449c823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:24.041287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 581/1168] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:17:24.308483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a2b87-5371-49d1-9329-655d048717f0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:24.421198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a2b87-5371-49d1-9329-655d048717f0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:24.455765 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8e759-b0eb-4ea0-bc05-8a4d0a3a1ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:24.456353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8e759-b0eb-4ea0-bc05-8a4d0a3a1ac5 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:24.456635 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8e759-b0eb-4ea0-bc05-8a4d0a3a1ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:24.456907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8e759-b0eb-4ea0-bc05-8a4d0a3a1ac5 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:24.457608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8e759-b0eb-4ea0-bc05-8a4d0a3a1ac5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:24.458966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8e759-b0eb-4ea0-bc05-8a4d0a3a1ac5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:24.478099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 588/1169] 10.222.0.60 () {60 vars in 1138 bytes} [Sat Sep 17 04:17:24 2022] POST /identity/v3/domains => generated 267 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:17:26.635669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acc7c8-1419-4977-a9c2-08d9b9f3ab33 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:26.636612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acc7c8-1419-4977-a9c2-08d9b9f3ab33 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:26.636612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acc7c8-1419-4977-a9c2-08d9b9f3ab33 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:26.638822 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 582/1170] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:26.644348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8dc47-a169-4282-88ea-ca6f600be072 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:26.644771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8dc47-a169-4282-88ea-ca6f600be072 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:26.644882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8dc47-a169-4282-88ea-ca6f600be072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:26.664269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3be8dc47-a169-4282-88ea-ca6f600be072 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:26.665486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3be8dc47-a169-4282-88ea-ca6f600be072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:26.678943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:26.735779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 589/1171] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:17:27.007670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acc7c8-1419-4977-a9c2-08d9b9f3ab33 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:27.115231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acc7c8-1419-4977-a9c2-08d9b9f3ab33 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:27.128206 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:27.152361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cabb4-6d23-46e4-921d-942188cf5ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:27.152985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cabb4-6d23-46e4-921d-942188cf5ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:27.153292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cabb4-6d23-46e4-921d-942188cf5ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:27.153557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cabb4-6d23-46e4-921d-942188cf5ae3 None admin] PATH_INFO: `/v3/domains/5b0011ce20394621a84da4d12ac8a130` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:27.164159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cabb4-6d23-46e4-921d-942188cf5ae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5b0011ce20394621a84da4d12ac8a130)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:27.165629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cabb4-6d23-46e4-921d-942188cf5ae3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:27.167183 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 583/1172] 10.222.0.60 () {56 vars in 1183 bytes} [Sat Sep 17 04:17:27 2022] GET /identity/v3/domains/5b0011ce20394621a84da4d12ac8a130 => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:27.173184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8dc47-a169-4282-88ea-ca6f600be072 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:27.271969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8dc47-a169-4282-88ea-ca6f600be072 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:27.306419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e7c64-4301-4722-91b1-3e59cf4253e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:27.306999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7c64-4301-4722-91b1-3e59cf4253e9 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:27.307266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7c64-4301-4722-91b1-3e59cf4253e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:27.307531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7c64-4301-4722-91b1-3e59cf4253e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:27.308172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e7c64-4301-4722-91b1-3e59cf4253e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:27.309628 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e7c64-4301-4722-91b1-3e59cf4253e9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:27.333375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:27.340418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 590/1173] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:17:27 2022] POST /identity/v3/projects => generated 361 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:17:29.496296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142aa52-5d04-46e9-9645-c7b39a5c8929 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:29.497195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142aa52-5d04-46e9-9645-c7b39a5c8929 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:29.497405 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142aa52-5d04-46e9-9645-c7b39a5c8929 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:29.499511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 584/1174] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:29.505965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43349d-2460-425e-914f-42f3e0262f0d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:29.506728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43349d-2460-425e-914f-42f3e0262f0d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:29.506914 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43349d-2460-425e-914f-42f3e0262f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:29.532431 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ac43349d-2460-425e-914f-42f3e0262f0d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:29.533676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ac43349d-2460-425e-914f-42f3e0262f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:29.605465 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 591/1175] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:17:29.872336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142aa52-5d04-46e9-9645-c7b39a5c8929 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:29.988518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142aa52-5d04-46e9-9645-c7b39a5c8929 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:30.024258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b500cea-ad4d-4c35-9a7d-c3704007a622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:30.024866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b500cea-ad4d-4c35-9a7d-c3704007a622 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:30.025138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b500cea-ad4d-4c35-9a7d-c3704007a622 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:30.025429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b500cea-ad4d-4c35-9a7d-c3704007a622 None admin] PATH_INFO: `/v3/domains/5b0011ce20394621a84da4d12ac8a130` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:30.026655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b500cea-ad4d-4c35-9a7d-c3704007a622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5b0011ce20394621a84da4d12ac8a130)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:30.028075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b500cea-ad4d-4c35-9a7d-c3704007a622 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:30.029656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 585/1176] 10.222.0.60 () {56 vars in 1183 bytes} [Sat Sep 17 04:17:29 2022] GET /identity/v3/domains/5b0011ce20394621a84da4d12ac8a130 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:30.038226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43349d-2460-425e-914f-42f3e0262f0d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:30.142966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43349d-2460-425e-914f-42f3e0262f0d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:30.174745 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:30.181949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1502040-ac52-4912-8c3a-6907ace1dff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:30.182543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1502040-ac52-4912-8c3a-6907ace1dff0 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:30.182821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1502040-ac52-4912-8c3a-6907ace1dff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:30.183078 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1502040-ac52-4912-8c3a-6907ace1dff0 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:30.183722 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1502040-ac52-4912-8c3a-6907ace1dff0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:30.185177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1502040-ac52-4912-8c3a-6907ace1dff0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:30.207105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 592/1177] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:17:30 2022] POST /identity/v3/users => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:17:32.362907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b717ce-974b-40b7-baaa-a8209a7203ed None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:32.363784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b717ce-974b-40b7-baaa-a8209a7203ed None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:32.363784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b717ce-974b-40b7-baaa-a8209a7203ed None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:32.366346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 586/1178] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:32.371883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde75266-c57d-4240-9c5e-da1fbc481591 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:32.372330 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde75266-c57d-4240-9c5e-da1fbc481591 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:32.372421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde75266-c57d-4240-9c5e-da1fbc481591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:32.390865 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dde75266-c57d-4240-9c5e-da1fbc481591 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:32.392015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dde75266-c57d-4240-9c5e-da1fbc481591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:32.459261 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 593/1179] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:17:32.721833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b717ce-974b-40b7-baaa-a8209a7203ed None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:32.772669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:32.833506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b717ce-974b-40b7-baaa-a8209a7203ed None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:32.868637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffadb8a-20c3-491a-9428-cdb958c7d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:32.869283 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffadb8a-20c3-491a-9428-cdb958c7d9cd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:32.869564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffadb8a-20c3-491a-9428-cdb958c7d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:32.869836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffadb8a-20c3-491a-9428-cdb958c7d9cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:32.880906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffadb8a-20c3-491a-9428-cdb958c7d9cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-cffadb8a-20c3-491a-9428-cdb958c7d9cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:17:32.885747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:17:32.886826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:17:32.887712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 587/1180] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:17:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:17:32.895329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde75266-c57d-4240-9c5e-da1fbc481591 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:33.003753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde75266-c57d-4240-9c5e-da1fbc481591 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:33.020042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:33.040392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d6ce4-3a07-44e6-83af-b3213d47c069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:33.040993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d6ce4-3a07-44e6-83af-b3213d47c069 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:33.041310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d6ce4-3a07-44e6-83af-b3213d47c069 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:33.041579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d6ce4-3a07-44e6-83af-b3213d47c069 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:33.042242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d6ce4-3a07-44e6-83af-b3213d47c069 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:17:33.042551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d6ce4-3a07-44e6-83af-b3213d47c069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:33.043912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d6ce4-3a07-44e6-83af-b3213d47c069 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:33.051982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 594/1181] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:17:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:33.058270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffadb8a-20c3-491a-9428-cdb958c7d9cd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:33.165166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:33.168485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffadb8a-20c3-491a-9428-cdb958c7d9cd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:33.208905 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42279ab9-9253-40ab-8b4c-a18ef3b49846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:33.209546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42279ab9-9253-40ab-8b4c-a18ef3b49846 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:33.209812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42279ab9-9253-40ab-8b4c-a18ef3b49846 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:33.210082 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42279ab9-9253-40ab-8b4c-a18ef3b49846 None admin] PATH_INFO: `/v3/users/1c8f9862be734c798d3d61bce583ae41` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:33.223610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42279ab9-9253-40ab-8b4c-a18ef3b49846 None admin] RBAC: Authorizing `identity:get_user(user_id=1c8f9862be734c798d3d61bce583ae41)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:33.225038 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42279ab9-9253-40ab-8b4c-a18ef3b49846 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:33.226658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 588/1182] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:17:33 2022] GET /identity/v3/users/1c8f9862be734c798d3d61bce583ae41 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:33.235923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d6ce4-3a07-44e6-83af-b3213d47c069 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:33.339228 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d6ce4-3a07-44e6-83af-b3213d47c069 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:33.376987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a780a1-2450-4789-8f81-0205b6fddbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:33.377664 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a780a1-2450-4789-8f81-0205b6fddbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:33.377892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a780a1-2450-4789-8f81-0205b6fddbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:33.378165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a780a1-2450-4789-8f81-0205b6fddbe2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:33.427318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:33.456067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a780a1-2450-4789-8f81-0205b6fddbe2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:33.458298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a780a1-2450-4789-8f81-0205b6fddbe2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:33.568860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 595/1183] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:17:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:17:33.575713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42279ab9-9253-40ab-8b4c-a18ef3b49846 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:33.685947 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42279ab9-9253-40ab-8b4c-a18ef3b49846 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:33.722412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de70a98-d75c-4160-b75b-c83830ba3a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:33.723051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de70a98-d75c-4160-b75b-c83830ba3a91 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:33.723326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de70a98-d75c-4160-b75b-c83830ba3a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:33.723594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de70a98-d75c-4160-b75b-c83830ba3a91 None admin] PATH_INFO: `/v3/projects/09690aa01792497fbbf72efe7730a415` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:33.724770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de70a98-d75c-4160-b75b-c83830ba3a91 None admin] RBAC: Authorizing `identity:get_project(project_id=09690aa01792497fbbf72efe7730a415)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:33.726257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de70a98-d75c-4160-b75b-c83830ba3a91 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:33.727978 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 589/1184] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:17:33 2022] GET /identity/v3/projects/09690aa01792497fbbf72efe7730a415 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:17:33.736852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a780a1-2450-4789-8f81-0205b6fddbe2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:33.820354 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:33.845531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a780a1-2450-4789-8f81-0205b6fddbe2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:33.883958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff51c1a-f91c-4ed4-a4c8-bbf9286aaab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:33.884555 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff51c1a-f91c-4ed4-a4c8-bbf9286aaab3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:33.884835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff51c1a-f91c-4ed4-a4c8-bbf9286aaab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:33.885121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff51c1a-f91c-4ed4-a4c8-bbf9286aaab3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:33.885853 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff51c1a-f91c-4ed4-a4c8-bbf9286aaab3 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=09690aa01792497fbbf72efe7730a415, user.id=1c8f9862be734c798d3d61bce583ae41) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:17:33.886158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff51c1a-f91c-4ed4-a4c8-bbf9286aaab3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:33.887604 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff51c1a-f91c-4ed4-a4c8-bbf9286aaab3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:33.893118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 596/1185] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:17:33 2022] GET /identity/v3/role_assignments?user.id=1c8f9862be734c798d3d61bce583ae41&scope.project.id=09690aa01792497fbbf72efe7730a415&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:36.006880 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a3071-ab0d-453c-bcf9-6073325e4853 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:36.007732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a3071-ab0d-453c-bcf9-6073325e4853 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:36.007732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a3071-ab0d-453c-bcf9-6073325e4853 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:36.009210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 590/1186] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:36.015398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255eb72-70be-408a-b290-35afe283d369 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:36.016095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255eb72-70be-408a-b290-35afe283d369 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:36.016242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255eb72-70be-408a-b290-35afe283d369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:36.037937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0255eb72-70be-408a-b290-35afe283d369 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:36.039134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0255eb72-70be-408a-b290-35afe283d369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:36.108706 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 597/1187] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:17:36.377011 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a3071-ab0d-453c-bcf9-6073325e4853 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:36.414690 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:36.507221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a3071-ab0d-453c-bcf9-6073325e4853 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:36.542339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756f588-33ea-4876-a2a9-18205c682f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:36.542957 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756f588-33ea-4876-a2a9-18205c682f3b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:36.543243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756f588-33ea-4876-a2a9-18205c682f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:36.543484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756f588-33ea-4876-a2a9-18205c682f3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:36.554305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756f588-33ea-4876-a2a9-18205c682f3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e756f588-33ea-4876-a2a9-18205c682f3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:17:36.559107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:17:36.560193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:17:36.561267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 591/1188] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:17:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:17:36.569126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255eb72-70be-408a-b290-35afe283d369 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:36.650385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:36.681546 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255eb72-70be-408a-b290-35afe283d369 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:36.716730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f61e65-1b91-4c9d-989f-bc107f39e4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:36.717368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f61e65-1b91-4c9d-989f-bc107f39e4ff None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:36.717648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f61e65-1b91-4c9d-989f-bc107f39e4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:36.717900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f61e65-1b91-4c9d-989f-bc107f39e4ff None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:36.718587 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f61e65-1b91-4c9d-989f-bc107f39e4ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:17:36.718887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f61e65-1b91-4c9d-989f-bc107f39e4ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:36.720255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f61e65-1b91-4c9d-989f-bc107f39e4ff None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:36.728206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 598/1189] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:17:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:36.734501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756f588-33ea-4876-a2a9-18205c682f3b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:36.850578 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:36.872811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756f588-33ea-4876-a2a9-18205c682f3b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:36.909760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba60f3a-d3b3-4526-8531-774333df4522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:36.910319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba60f3a-d3b3-4526-8531-774333df4522 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:36.910596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba60f3a-d3b3-4526-8531-774333df4522 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:36.910866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba60f3a-d3b3-4526-8531-774333df4522 None admin] PATH_INFO: `/v3/users/1c8f9862be734c798d3d61bce583ae41` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:36.912118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba60f3a-d3b3-4526-8531-774333df4522 None admin] RBAC: Authorizing `identity:get_user(user_id=1c8f9862be734c798d3d61bce583ae41)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:36.913590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba60f3a-d3b3-4526-8531-774333df4522 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:36.915180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 592/1190] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:17:36 2022] GET /identity/v3/users/1c8f9862be734c798d3d61bce583ae41 => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:36.924725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f61e65-1b91-4c9d-989f-bc107f39e4ff None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:37.029963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f61e65-1b91-4c9d-989f-bc107f39e4ff None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:37.064406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bddc515-0503-49eb-9f41-4c279e91821f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:37.065016 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddc515-0503-49eb-9f41-4c279e91821f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:37.065319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddc515-0503-49eb-9f41-4c279e91821f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:37.065588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddc515-0503-49eb-9f41-4c279e91821f None admin] PATH_INFO: `/v3/projects/09690aa01792497fbbf72efe7730a415` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:37.066808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bddc515-0503-49eb-9f41-4c279e91821f None admin] RBAC: Authorizing `identity:get_project(project_id=09690aa01792497fbbf72efe7730a415)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:37.068225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bddc515-0503-49eb-9f41-4c279e91821f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:37.069871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 599/1191] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:17:36 2022] GET /identity/v3/projects/09690aa01792497fbbf72efe7730a415 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:37.076728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba60f3a-d3b3-4526-8531-774333df4522 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:37.190420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba60f3a-d3b3-4526-8531-774333df4522 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:37.230711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c11912-6b40-4ad7-a7cd-ebe3b8ef3c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:37.231364 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c11912-6b40-4ad7-a7cd-ebe3b8ef3c37 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:37.231637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c11912-6b40-4ad7-a7cd-ebe3b8ef3c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:37.231906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c11912-6b40-4ad7-a7cd-ebe3b8ef3c37 None admin] PATH_INFO: `/v3/projects/09690aa01792497fbbf72efe7730a415/users/1c8f9862be734c798d3d61bce583ae41/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:37.234079 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c11912-6b40-4ad7-a7cd-ebe3b8ef3c37 None admin] RBAC: Authorizing `identity:create_grant(project_id=09690aa01792497fbbf72efe7730a415, user_id=1c8f9862be734c798d3d61bce583ae41, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:37.235507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c11912-6b40-4ad7-a7cd-ebe3b8ef3c37 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:37.254476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 593/1192] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:17:37 2022] PUT /identity/v3/projects/09690aa01792497fbbf72efe7730a415/users/1c8f9862be734c798d3d61bce583ae41/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:17:39.393925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121255d-6da4-4b4f-a79f-2007814fd4d6 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:39.394491 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121255d-6da4-4b4f-a79f-2007814fd4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:39.394491 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121255d-6da4-4b4f-a79f-2007814fd4d6 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:39.396496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 600/1193] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:17:39.402958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1d15b-325a-4bfa-b4fb-97df22883b3e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:39.403404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1d15b-325a-4bfa-b4fb-97df22883b3e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:39.403493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1d15b-325a-4bfa-b4fb-97df22883b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:39.422783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-10e1d15b-325a-4bfa-b4fb-97df22883b3e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:17:39.423948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-10e1d15b-325a-4bfa-b4fb-97df22883b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:17:39.433122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:39.507592 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 594/1194] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:17:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 04:17:39.771694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121255d-6da4-4b4f-a79f-2007814fd4d6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:39.776596 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:39.879367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121255d-6da4-4b4f-a79f-2007814fd4d6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:39.914343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048dda7-1030-4c5d-bce1-4f3a90f6bf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:39.914960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048dda7-1030-4c5d-bce1-4f3a90f6bf3e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:39.915233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048dda7-1030-4c5d-bce1-4f3a90f6bf3e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:39.915500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048dda7-1030-4c5d-bce1-4f3a90f6bf3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:39.927022 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048dda7-1030-4c5d-bce1-4f3a90f6bf3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8048dda7-1030-4c5d-bce1-4f3a90f6bf3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:17:39.931884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:17:39.932935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:17:39.933839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 601/1195] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:17:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:17:39.939162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1d15b-325a-4bfa-b4fb-97df22883b3e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:40.061677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1d15b-325a-4bfa-b4fb-97df22883b3e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:40.098300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4964d1-e04d-4b17-bff2-3d084f8062c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:40.098896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4964d1-e04d-4b17-bff2-3d084f8062c7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:40.099171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4964d1-e04d-4b17-bff2-3d084f8062c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:40.099439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4964d1-e04d-4b17-bff2-3d084f8062c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:40.100101 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4964d1-e04d-4b17-bff2-3d084f8062c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:17:40.100400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4964d1-e04d-4b17-bff2-3d084f8062c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:40.101830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4964d1-e04d-4b17-bff2-3d084f8062c7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:40.109980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 595/1196] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:17:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:40.116457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048dda7-1030-4c5d-bce1-4f3a90f6bf3e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:40.183910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:40.223395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048dda7-1030-4c5d-bce1-4f3a90f6bf3e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:40.260888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538efa8-d3c5-461e-8dbc-7b83e6c311b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:40.261534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538efa8-d3c5-461e-8dbc-7b83e6c311b1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:40.261810 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538efa8-d3c5-461e-8dbc-7b83e6c311b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:40.262077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538efa8-d3c5-461e-8dbc-7b83e6c311b1 None admin] PATH_INFO: `/v3/users/1c8f9862be734c798d3d61bce583ae41` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:40.263250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538efa8-d3c5-461e-8dbc-7b83e6c311b1 None admin] RBAC: Authorizing `identity:get_user(user_id=1c8f9862be734c798d3d61bce583ae41)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:40.264674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538efa8-d3c5-461e-8dbc-7b83e6c311b1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:40.266262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 602/1197] 10.222.0.60 () {56 vars in 1177 bytes} [Sat Sep 17 04:17:40 2022] GET /identity/v3/users/1c8f9862be734c798d3d61bce583ae41 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:40.272100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4964d1-e04d-4b17-bff2-3d084f8062c7 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:40.282309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:40.376902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4964d1-e04d-4b17-bff2-3d084f8062c7 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:40.411571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034e505-0624-4f0e-8f44-2efc9da2cb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:40.412156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034e505-0624-4f0e-8f44-2efc9da2cb63 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:40.412424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034e505-0624-4f0e-8f44-2efc9da2cb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:40.412692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034e505-0624-4f0e-8f44-2efc9da2cb63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:40.479965 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034e505-0624-4f0e-8f44-2efc9da2cb63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:40.481354 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034e505-0624-4f0e-8f44-2efc9da2cb63 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:40.488810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:40.606130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 596/1198] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:17:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:17:40.612494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538efa8-d3c5-461e-8dbc-7b83e6c311b1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:40.722688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538efa8-d3c5-461e-8dbc-7b83e6c311b1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:40.751536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:40.758660 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533a44b-03ca-4c00-b5db-be79c61e8ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:40.759288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a44b-03ca-4c00-b5db-be79c61e8ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:40.759915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a44b-03ca-4c00-b5db-be79c61e8ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:40.760184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a44b-03ca-4c00-b5db-be79c61e8ca7 None admin] PATH_INFO: `/v3/projects/09690aa01792497fbbf72efe7730a415` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:40.761396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533a44b-03ca-4c00-b5db-be79c61e8ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=09690aa01792497fbbf72efe7730a415)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:40.762841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533a44b-03ca-4c00-b5db-be79c61e8ca7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:40.764468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 603/1199] 10.222.0.60 () {56 vars in 1186 bytes} [Sat Sep 17 04:17:40 2022] GET /identity/v3/projects/09690aa01792497fbbf72efe7730a415 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:40.774246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034e505-0624-4f0e-8f44-2efc9da2cb63 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:17:40.875811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034e505-0624-4f0e-8f44-2efc9da2cb63 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:17:40.884066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:40.922775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e893a-d478-4de3-9058-e5422d06b892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:17:40.923473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e893a-d478-4de3-9058-e5422d06b892 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:17:40.923750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e893a-d478-4de3-9058-e5422d06b892 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:17:40.924052 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e893a-d478-4de3-9058-e5422d06b892 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:17:40.925201 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e893a-d478-4de3-9058-e5422d06b892 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=09690aa01792497fbbf72efe7730a415, user.id=1c8f9862be734c798d3d61bce583ae41) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:17:40.925666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e893a-d478-4de3-9058-e5422d06b892 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:17:40.927898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e893a-d478-4de3-9058-e5422d06b892 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:17:40.937504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 597/1200] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:17:40 2022] GET /identity/v3/role_assignments?user.id=1c8f9862be734c798d3d61bce583ae41&scope.project.id=09690aa01792497fbbf72efe7730a415&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:17:50.755521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:17:50.932844 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=15 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:36.450871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5799e7e-39ee-47b8-aca8-aba2d98559c4 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:36.450871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5799e7e-39ee-47b8-aca8-aba2d98559c4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:36.450871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5799e7e-39ee-47b8-aca8-aba2d98559c4 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:36.450871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 604/1201] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:18:36.457460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e53058-b448-49cd-8dc7-d1911bc7f452 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:36.458382 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e53058-b448-49cd-8dc7-d1911bc7f452 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:36.458867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e53058-b448-49cd-8dc7-d1911bc7f452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:36.480116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-73e53058-b448-49cd-8dc7-d1911bc7f452 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:18:36.481704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-73e53058-b448-49cd-8dc7-d1911bc7f452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:18:36.561667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 598/1202] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:18:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:18:36.828590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5799e7e-39ee-47b8-aca8-aba2d98559c4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:36.947711 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5799e7e-39ee-47b8-aca8-aba2d98559c4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:36.991113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6255994-2de8-408b-ba1c-c6c00badc3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:36.991749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6255994-2de8-408b-ba1c-c6c00badc3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:36.992127 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6255994-2de8-408b-ba1c-c6c00badc3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:36.992398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6255994-2de8-408b-ba1c-c6c00badc3b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:37.004825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6255994-2de8-408b-ba1c-c6c00badc3b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:37.006499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6255994-2de8-408b-ba1c-c6c00badc3b9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e6255994-2de8-408b-ba1c-c6c00badc3b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:18:37.023235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:37.024637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:18:37.025695 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:18:37.025695 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:37.025695 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:18:37.025695 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 605/1203] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:37.030774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e53058-b448-49cd-8dc7-d1911bc7f452 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:37.152239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e53058-b448-49cd-8dc7-d1911bc7f452 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:37.187308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:37.195260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc62d1d-364c-4592-b0dd-e8a0636037d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:37.195930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62d1d-364c-4592-b0dd-e8a0636037d1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:37.196247 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62d1d-364c-4592-b0dd-e8a0636037d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:37.196497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62d1d-364c-4592-b0dd-e8a0636037d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:37.197241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc62d1d-364c-4592-b0dd-e8a0636037d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:37.197569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc62d1d-364c-4592-b0dd-e8a0636037d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:37.198924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc62d1d-364c-4592-b0dd-e8a0636037d1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:37.213370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 599/1204] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:37.219206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6255994-2de8-408b-ba1c-c6c00badc3b9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:37.283149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:37.338897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6255994-2de8-408b-ba1c-c6c00badc3b9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:37.382749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4332c-b162-4098-86de-ceaac5a29581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:37.383440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4332c-b162-4098-86de-ceaac5a29581 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:37.383689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4332c-b162-4098-86de-ceaac5a29581 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:37.383996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4332c-b162-4098-86de-ceaac5a29581 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:37.384658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4332c-b162-4098-86de-ceaac5a29581 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:37.386220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4332c-b162-4098-86de-ceaac5a29581 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:37.421207 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 606/1205] 10.222.0.60 () {60 vars in 1132 bytes} [Sat Sep 17 04:18:37 2022] POST /identity/v3/users => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:18:39.735686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc8691-84f4-494a-904f-c57f0a65dca0 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:39.736315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc8691-84f4-494a-904f-c57f0a65dca0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:39.736315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc8691-84f4-494a-904f-c57f0a65dca0 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:39.737712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 600/1206] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:18:39.744270 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822047e8-0637-44af-8739-e92fd3602f83 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:39.744911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822047e8-0637-44af-8739-e92fd3602f83 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:39.744911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822047e8-0637-44af-8739-e92fd3602f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:39.765431 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-822047e8-0637-44af-8739-e92fd3602f83 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:18:39.766650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-822047e8-0637-44af-8739-e92fd3602f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:18:39.847204 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 607/1207] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:18:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:18:40.120035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc8691-84f4-494a-904f-c57f0a65dca0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:40.228786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc8691-84f4-494a-904f-c57f0a65dca0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:40.266055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c80e28-2304-4f06-835c-589c794c8897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:40.266683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c80e28-2304-4f06-835c-589c794c8897 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:40.267033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c80e28-2304-4f06-835c-589c794c8897 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:40.267247 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c80e28-2304-4f06-835c-589c794c8897 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:40.279973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c80e28-2304-4f06-835c-589c794c8897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-93c80e28-2304-4f06-835c-589c794c8897 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:18:40.284961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:40.285993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:18:40.287264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 601/1208] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:18:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:40.292251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822047e8-0637-44af-8739-e92fd3602f83 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:40.334965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:40.402745 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822047e8-0637-44af-8739-e92fd3602f83 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:40.439118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc742cb-91ad-4cc3-a9f9-9ec561f04eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:40.439829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc742cb-91ad-4cc3-a9f9-9ec561f04eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:40.440174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc742cb-91ad-4cc3-a9f9-9ec561f04eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:40.440512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc742cb-91ad-4cc3-a9f9-9ec561f04eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:40.441423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc742cb-91ad-4cc3-a9f9-9ec561f04eb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:40.441788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc742cb-91ad-4cc3-a9f9-9ec561f04eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:40.443222 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc742cb-91ad-4cc3-a9f9-9ec561f04eb0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:40.454019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 608/1209] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:18:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:40.460297 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c80e28-2304-4f06-835c-589c794c8897 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:40.516091 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:40.573497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c80e28-2304-4f06-835c-589c794c8897 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:40.611477 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0d8ba-77d6-4b69-a9f3-6117a81e6afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:40.612334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0d8ba-77d6-4b69-a9f3-6117a81e6afb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:40.612723 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0d8ba-77d6-4b69-a9f3-6117a81e6afb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:40.613066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0d8ba-77d6-4b69-a9f3-6117a81e6afb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:40.625409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0d8ba-77d6-4b69-a9f3-6117a81e6afb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:40.626898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0d8ba-77d6-4b69-a9f3-6117a81e6afb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-20f0d8ba-77d6-4b69-a9f3-6117a81e6afb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:18:40.644110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:40.645285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:18:40.646363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:18:40.646363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:40.646363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:18:40.646363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 602/1210] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:40.651537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc742cb-91ad-4cc3-a9f9-9ec561f04eb0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:40.761615 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:40.764705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc742cb-91ad-4cc3-a9f9-9ec561f04eb0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:40.799508 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef489f-f1be-49d0-80bc-9aaae075feeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:40.800119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef489f-f1be-49d0-80bc-9aaae075feeb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:40.800483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef489f-f1be-49d0-80bc-9aaae075feeb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:40.800649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef489f-f1be-49d0-80bc-9aaae075feeb None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:40.801320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef489f-f1be-49d0-80bc-9aaae075feeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:40.801615 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef489f-f1be-49d0-80bc-9aaae075feeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:40.802978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef489f-f1be-49d0-80bc-9aaae075feeb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:40.816979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 609/1211] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:40.822847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0d8ba-77d6-4b69-a9f3-6117a81e6afb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:40.935391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0d8ba-77d6-4b69-a9f3-6117a81e6afb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:40.937319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:40.974714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d17f8-5524-4f1a-998e-1bb88b52a6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:40.975578 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d17f8-5524-4f1a-998e-1bb88b52a6da None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:40.975953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d17f8-5524-4f1a-998e-1bb88b52a6da None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:40.976369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d17f8-5524-4f1a-998e-1bb88b52a6da None admin] PATH_INFO: `/v3/users/placement` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:40.985454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d17f8-5524-4f1a-998e-1bb88b52a6da None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:40.987132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d17f8-5524-4f1a-998e-1bb88b52a6da None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-330d17f8-5524-4f1a-998e-1bb88b52a6da None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:18:41.000493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:41.001877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:41.003075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:41.003075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:41.003075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:18:41.003075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:18:41.003075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:18:41.003075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:18:41.003075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:18:41.003075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:18:41.003075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 04:18:41.003075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:18:41.004780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 603/1212] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:41.010872 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef489f-f1be-49d0-80bc-9aaae075feeb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:41.129301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef489f-f1be-49d0-80bc-9aaae075feeb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:41.174120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed78c0-7f5a-47b0-b776-7c1e98295b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:41.174858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed78c0-7f5a-47b0-b776-7c1e98295b24 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:41.175506 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed78c0-7f5a-47b0-b776-7c1e98295b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:41.175780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed78c0-7f5a-47b0-b776-7c1e98295b24 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:41.176548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed78c0-7f5a-47b0-b776-7c1e98295b24 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:41.176851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed78c0-7f5a-47b0-b776-7c1e98295b24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:41.178328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed78c0-7f5a-47b0-b776-7c1e98295b24 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:41.198450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 610/1213] 10.222.0.60 () {56 vars in 1143 bytes} [Sat Sep 17 04:18:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:41.204453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d17f8-5524-4f1a-998e-1bb88b52a6da None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:41.310544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d17f8-5524-4f1a-998e-1bb88b52a6da None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:41.348212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8bf5a-f14b-4935-b8ff-7b8ada09415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:41.348844 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8bf5a-f14b-4935-b8ff-7b8ada09415a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:41.349208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8bf5a-f14b-4935-b8ff-7b8ada09415a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:41.349416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8bf5a-f14b-4935-b8ff-7b8ada09415a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:41.386118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:41.430966 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8bf5a-f14b-4935-b8ff-7b8ada09415a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:41.432414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8bf5a-f14b-4935-b8ff-7b8ada09415a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:41.546258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 604/1214] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:18:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:18:41.552148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed78c0-7f5a-47b0-b776-7c1e98295b24 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:41.572169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:41.666280 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed78c0-7f5a-47b0-b776-7c1e98295b24 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:41.707245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418b385-e99a-4ece-8513-d705ed6d3ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:41.708047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418b385-e99a-4ece-8513-d705ed6d3ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:41.708355 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418b385-e99a-4ece-8513-d705ed6d3ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:41.708689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418b385-e99a-4ece-8513-d705ed6d3ea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:41.721758 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418b385-e99a-4ece-8513-d705ed6d3ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:41.723256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418b385-e99a-4ece-8513-d705ed6d3ea9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c418b385-e99a-4ece-8513-d705ed6d3ea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:18:41.738738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:41.739769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:18:41.745609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:18:41.745609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:41.745609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:18:41.745609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 611/1215] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:18:41.745844 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8bf5a-f14b-4935-b8ff-7b8ada09415a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:41.819410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:41.865005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8bf5a-f14b-4935-b8ff-7b8ada09415a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:41.903893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dd294-0d7a-4d0b-9eb0-f2810e92078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:41.904607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dd294-0d7a-4d0b-9eb0-f2810e92078a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:41.904904 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dd294-0d7a-4d0b-9eb0-f2810e92078a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:41.905358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dd294-0d7a-4d0b-9eb0-f2810e92078a None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:41.906200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876dd294-0d7a-4d0b-9eb0-f2810e92078a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:41.906564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876dd294-0d7a-4d0b-9eb0-f2810e92078a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:41.908685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876dd294-0d7a-4d0b-9eb0-f2810e92078a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:41.925328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 605/1216] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:41.931406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418b385-e99a-4ece-8513-d705ed6d3ea9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:42.013524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:42.049861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418b385-e99a-4ece-8513-d705ed6d3ea9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:42.087208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e392551-25c6-474c-a2e8-7bc1c073c285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:42.087776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e392551-25c6-474c-a2e8-7bc1c073c285 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:42.088048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e392551-25c6-474c-a2e8-7bc1c073c285 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:42.088311 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e392551-25c6-474c-a2e8-7bc1c073c285 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:42.097139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e392551-25c6-474c-a2e8-7bc1c073c285 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:42.098632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e392551-25c6-474c-a2e8-7bc1c073c285 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5e392551-25c6-474c-a2e8-7bc1c073c285 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:18:42.112292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:42.113713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:42.114616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:18:42.114616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:18:42.114616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:18:42.114616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:18:42.114616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:18:42.114616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:18:42.114616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:18:42.114616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:18:42.114616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 612/1217] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:18:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:42.120375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dd294-0d7a-4d0b-9eb0-f2810e92078a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:42.248355 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dd294-0d7a-4d0b-9eb0-f2810e92078a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:42.279988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:42.294711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9d162-b14b-4a68-8db9-d6e12f8ebae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:42.295408 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9d162-b14b-4a68-8db9-d6e12f8ebae5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:42.295727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9d162-b14b-4a68-8db9-d6e12f8ebae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:42.296001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9d162-b14b-4a68-8db9-d6e12f8ebae5 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:42.296790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9d162-b14b-4a68-8db9-d6e12f8ebae5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:42.297075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9d162-b14b-4a68-8db9-d6e12f8ebae5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:42.298571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9d162-b14b-4a68-8db9-d6e12f8ebae5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:42.311534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 606/1218] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:18:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:42.317833 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e392551-25c6-474c-a2e8-7bc1c073c285 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:42.435244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e392551-25c6-474c-a2e8-7bc1c073c285 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:42.464174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:42.470545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21df8b8-5bdb-4f6e-aad1-d5bb0faeb56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:42.471119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21df8b8-5bdb-4f6e-aad1-d5bb0faeb56d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:42.471392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21df8b8-5bdb-4f6e-aad1-d5bb0faeb56d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:42.471658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21df8b8-5bdb-4f6e-aad1-d5bb0faeb56d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:42.472384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21df8b8-5bdb-4f6e-aad1-d5bb0faeb56d None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=e6d3b7887c13466ab8e3d9737f907c56) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:42.472681 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21df8b8-5bdb-4f6e-aad1-d5bb0faeb56d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:42.474190 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21df8b8-5bdb-4f6e-aad1-d5bb0faeb56d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:42.479976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 613/1219] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:18:42 2022] GET /identity/v3/role_assignments?user.id=e6d3b7887c13466ab8e3d9737f907c56&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:44.651069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacec67-1baf-456f-a662-3f1b06c47a28 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:44.651648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacec67-1baf-456f-a662-3f1b06c47a28 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:44.651648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacec67-1baf-456f-a662-3f1b06c47a28 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:44.653263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 607/1220] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:18:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:18:44.658913 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ae9f5-24c2-41a6-9c31-b85ab4c25698 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:44.659380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ae9f5-24c2-41a6-9c31-b85ab4c25698 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:44.659458 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ae9f5-24c2-41a6-9c31-b85ab4c25698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:44.678874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fa4ae9f5-24c2-41a6-9c31-b85ab4c25698 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:18:44.680111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fa4ae9f5-24c2-41a6-9c31-b85ab4c25698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:18:44.747795 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 614/1221] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:18:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:18:45.044078 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacec67-1baf-456f-a662-3f1b06c47a28 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:45.152833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacec67-1baf-456f-a662-3f1b06c47a28 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:45.187885 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3deaf94-8965-4127-876a-80721e58a636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:45.188520 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3deaf94-8965-4127-876a-80721e58a636 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:45.188819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3deaf94-8965-4127-876a-80721e58a636 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:45.189015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3deaf94-8965-4127-876a-80721e58a636 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:45.200256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3deaf94-8965-4127-876a-80721e58a636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-a3deaf94-8965-4127-876a-80721e58a636 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:18:45.205559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:45.207015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:18:45.207896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 608/1222] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:18:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:45.213394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ae9f5-24c2-41a6-9c31-b85ab4c25698 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:45.315810 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ae9f5-24c2-41a6-9c31-b85ab4c25698 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:45.341238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:45.358496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd92f0f-9cb0-4d91-873c-cad55f1b49b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:45.359137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd92f0f-9cb0-4d91-873c-cad55f1b49b0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:45.359475 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd92f0f-9cb0-4d91-873c-cad55f1b49b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:45.359787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd92f0f-9cb0-4d91-873c-cad55f1b49b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:45.360529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd92f0f-9cb0-4d91-873c-cad55f1b49b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:45.360871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd92f0f-9cb0-4d91-873c-cad55f1b49b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:45.362432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd92f0f-9cb0-4d91-873c-cad55f1b49b0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:45.372449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 615/1223] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:18:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:45.378718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3deaf94-8965-4127-876a-80721e58a636 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:45.427949 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:45.489560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3deaf94-8965-4127-876a-80721e58a636 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:45.525053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68852dd8-571d-4293-a1a1-91dededc43ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:45.525753 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68852dd8-571d-4293-a1a1-91dededc43ea None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:45.525995 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68852dd8-571d-4293-a1a1-91dededc43ea None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:45.526311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68852dd8-571d-4293-a1a1-91dededc43ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:45.537040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68852dd8-571d-4293-a1a1-91dededc43ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:45.538505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68852dd8-571d-4293-a1a1-91dededc43ea None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-68852dd8-571d-4293-a1a1-91dededc43ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:18:45.553227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:45.554479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:18:45.560411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:18:45.560411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:45.560411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:18:45.560411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 609/1224] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:45.560702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd92f0f-9cb0-4d91-873c-cad55f1b49b0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:45.665397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd92f0f-9cb0-4d91-873c-cad55f1b49b0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:45.702230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d3b60-a9d2-46e4-b650-4e27511be18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:45.703035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d3b60-a9d2-46e4-b650-4e27511be18b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:45.703370 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d3b60-a9d2-46e4-b650-4e27511be18b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:45.703651 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d3b60-a9d2-46e4-b650-4e27511be18b None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:45.704327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d3b60-a9d2-46e4-b650-4e27511be18b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:45.704655 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d3b60-a9d2-46e4-b650-4e27511be18b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:45.706159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d3b60-a9d2-46e4-b650-4e27511be18b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:45.718369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 616/1225] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:45.724405 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68852dd8-571d-4293-a1a1-91dededc43ea None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:45.823492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:45.830588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68852dd8-571d-4293-a1a1-91dededc43ea None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:45.868085 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d994bd-634b-4ec7-990f-f00758ca7f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:45.868945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d994bd-634b-4ec7-990f-f00758ca7f07 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:45.869289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d994bd-634b-4ec7-990f-f00758ca7f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:45.869619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d994bd-634b-4ec7-990f-f00758ca7f07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:45.877877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d994bd-634b-4ec7-990f-f00758ca7f07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:45.879308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d994bd-634b-4ec7-990f-f00758ca7f07 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c6d994bd-634b-4ec7-990f-f00758ca7f07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:18:45.892284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:45.893530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:45.894594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:45.894594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:45.894594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:18:45.894594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:18:45.894594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:18:45.894594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:18:45.894594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:18:45.894594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:18:45.894594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 04:18:45.894594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:18:45.894594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 610/1226] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:45.899841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d3b60-a9d2-46e4-b650-4e27511be18b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:45.951195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:46.009041 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d3b60-a9d2-46e4-b650-4e27511be18b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:46.043226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e8b52-408f-4797-9536-1cc836a39110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:46.043839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e8b52-408f-4797-9536-1cc836a39110 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:46.044117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e8b52-408f-4797-9536-1cc836a39110 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:46.044395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e8b52-408f-4797-9536-1cc836a39110 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:46.045244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e8b52-408f-4797-9536-1cc836a39110 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:46.045452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e8b52-408f-4797-9536-1cc836a39110 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:46.047127 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e8b52-408f-4797-9536-1cc836a39110 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:46.065727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 617/1227] 10.222.0.60 () {56 vars in 1143 bytes} [Sat Sep 17 04:18:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:46.072179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d994bd-634b-4ec7-990f-f00758ca7f07 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:46.179305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d994bd-634b-4ec7-990f-f00758ca7f07 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:46.216707 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8d041-a778-4bf3-bad8-929340827ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:46.217310 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8d041-a778-4bf3-bad8-929340827ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:46.217591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8d041-a778-4bf3-bad8-929340827ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:46.217862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8d041-a778-4bf3-bad8-929340827ad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:46.229185 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8d041-a778-4bf3-bad8-929340827ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:46.230646 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8d041-a778-4bf3-bad8-929340827ad2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-31e8d041-a778-4bf3-bad8-929340827ad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:18:46.250414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:46.251607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:18:46.252737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:18:46.252737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:46.252737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:18:46.252737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 611/1228] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:18:46.257998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e8b52-408f-4797-9536-1cc836a39110 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:46.364445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e8b52-408f-4797-9536-1cc836a39110 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:46.366843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:46.409367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3217ff-7c1d-4715-8a42-596b7a88782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:46.409982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3217ff-7c1d-4715-8a42-596b7a88782c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:46.410270 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3217ff-7c1d-4715-8a42-596b7a88782c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:46.410527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3217ff-7c1d-4715-8a42-596b7a88782c None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:46.411172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3217ff-7c1d-4715-8a42-596b7a88782c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:46.411475 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3217ff-7c1d-4715-8a42-596b7a88782c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:46.412874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3217ff-7c1d-4715-8a42-596b7a88782c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:46.424737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 618/1229] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:46.430799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8d041-a778-4bf3-bad8-929340827ad2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:46.440309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:46.538213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8d041-a778-4bf3-bad8-929340827ad2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:46.578324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fd429-2545-4837-95e5-dacaaf27e34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:46.578961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fd429-2545-4837-95e5-dacaaf27e34c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:46.579244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fd429-2545-4837-95e5-dacaaf27e34c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:46.579515 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fd429-2545-4837-95e5-dacaaf27e34c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:46.587497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182fd429-2545-4837-95e5-dacaaf27e34c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:46.589463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182fd429-2545-4837-95e5-dacaaf27e34c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-182fd429-2545-4837-95e5-dacaaf27e34c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:18:46.603781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:46.604872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:46.606357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:18:46.606357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:18:46.606357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:18:46.606357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:18:46.606357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:18:46.606357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:18:46.606357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:18:46.606357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:18:46.606357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 612/1230] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:18:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:46.611783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3217ff-7c1d-4715-8a42-596b7a88782c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:46.722899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3217ff-7c1d-4715-8a42-596b7a88782c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:46.764978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11152e19-627b-4a3e-81e0-6621e41c69f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:46.765623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11152e19-627b-4a3e-81e0-6621e41c69f6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:46.765901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11152e19-627b-4a3e-81e0-6621e41c69f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:46.766175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11152e19-627b-4a3e-81e0-6621e41c69f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:46.766869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11152e19-627b-4a3e-81e0-6621e41c69f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:46.767171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11152e19-627b-4a3e-81e0-6621e41c69f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:46.768599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11152e19-627b-4a3e-81e0-6621e41c69f6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:46.781795 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 619/1231] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:46.789491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fd429-2545-4837-95e5-dacaaf27e34c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:46.862171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:46.899832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fd429-2545-4837-95e5-dacaaf27e34c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:46.934527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08353b2f-5850-4ef8-b0c9-0edf5cfbffe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:46.935192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08353b2f-5850-4ef8-b0c9-0edf5cfbffe3 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:46.935469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08353b2f-5850-4ef8-b0c9-0edf5cfbffe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:46.935743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08353b2f-5850-4ef8-b0c9-0edf5cfbffe3 None admin] PATH_INFO: `/v3/projects/58466195aca94b188d82c1b3a647e767/users/e6d3b7887c13466ab8e3d9737f907c56/roles/3eccbe0521ef44d88574b9acd9f988db` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:46.951069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08353b2f-5850-4ef8-b0c9-0edf5cfbffe3 None admin] RBAC: Authorizing `identity:create_grant(project_id=58466195aca94b188d82c1b3a647e767, user_id=e6d3b7887c13466ab8e3d9737f907c56, role_id=3eccbe0521ef44d88574b9acd9f988db)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:46.952552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08353b2f-5850-4ef8-b0c9-0edf5cfbffe3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:46.964776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 613/1232] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:18:46 2022] PUT /identity/v3/projects/58466195aca94b188d82c1b3a647e767/users/e6d3b7887c13466ab8e3d9737f907c56/roles/3eccbe0521ef44d88574b9acd9f988db => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:18:49.084499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c61e68-7e19-4e45-9967-0d0aeec79499 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:49.085138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c61e68-7e19-4e45-9967-0d0aeec79499 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:49.085138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c61e68-7e19-4e45-9967-0d0aeec79499 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:49.086563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 620/1233] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:18:49.096129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c933e4-849c-4b16-91d8-646b7720b0fe None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:49.096129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c933e4-849c-4b16-91d8-646b7720b0fe None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:49.096129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c933e4-849c-4b16-91d8-646b7720b0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:49.111936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-85c933e4-849c-4b16-91d8-646b7720b0fe None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:18:49.113488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-85c933e4-849c-4b16-91d8-646b7720b0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:18:49.185004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 614/1234] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:18:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:18:49.448408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c61e68-7e19-4e45-9967-0d0aeec79499 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:49.475802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:49.565516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c61e68-7e19-4e45-9967-0d0aeec79499 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:49.602460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d9d64-c364-4df8-9452-42097f8658cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:49.603118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d9d64-c364-4df8-9452-42097f8658cb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:49.603394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d9d64-c364-4df8-9452-42097f8658cb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:49.603660 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d9d64-c364-4df8-9452-42097f8658cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:49.617235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d9d64-c364-4df8-9452-42097f8658cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3a5d9d64-c364-4df8-9452-42097f8658cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:18:49.621992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:49.625288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:18:49.626435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 621/1235] 10.222.0.60 () {56 vars in 1102 bytes} [Sat Sep 17 04:18:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:18:49.629268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c933e4-849c-4b16-91d8-646b7720b0fe None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:49.679192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:49.736938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c933e4-849c-4b16-91d8-646b7720b0fe None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:49.774282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e8f77-a2c7-40e7-9542-3cf9b698ec52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:49.775022 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e8f77-a2c7-40e7-9542-3cf9b698ec52 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:49.775335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e8f77-a2c7-40e7-9542-3cf9b698ec52 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:49.775627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e8f77-a2c7-40e7-9542-3cf9b698ec52 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:49.776348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e8f77-a2c7-40e7-9542-3cf9b698ec52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:49.776731 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e8f77-a2c7-40e7-9542-3cf9b698ec52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:49.778324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e8f77-a2c7-40e7-9542-3cf9b698ec52 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:49.787750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 615/1236] 10.222.0.60 () {56 vars in 1103 bytes} [Sat Sep 17 04:18:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:49.794341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d9d64-c364-4df8-9452-42097f8658cb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:49.883354 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:49.902185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d9d64-c364-4df8-9452-42097f8658cb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:49.939709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e7d67-d2b2-4f96-a41a-d52ac09ab141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:49.940352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e7d67-d2b2-4f96-a41a-d52ac09ab141 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:49.940627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e7d67-d2b2-4f96-a41a-d52ac09ab141 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:49.940902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e7d67-d2b2-4f96-a41a-d52ac09ab141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:49.951602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e7d67-d2b2-4f96-a41a-d52ac09ab141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:49.953411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e7d67-d2b2-4f96-a41a-d52ac09ab141 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-737e7d67-d2b2-4f96-a41a-d52ac09ab141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:18:49.969683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:49.970990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:18:49.972004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:18:49.972004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:49.972004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:18:49.972004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 622/1237] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:49.977064 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e8f77-a2c7-40e7-9542-3cf9b698ec52 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:50.085628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e8f77-a2c7-40e7-9542-3cf9b698ec52 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:50.095917 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:50.124330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe64e4-feaf-4bcb-a534-7f40b6c9e36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:50.125067 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe64e4-feaf-4bcb-a534-7f40b6c9e36f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:50.125312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe64e4-feaf-4bcb-a534-7f40b6c9e36f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:50.125626 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe64e4-feaf-4bcb-a534-7f40b6c9e36f None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:50.126479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe64e4-feaf-4bcb-a534-7f40b6c9e36f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:50.126819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe64e4-feaf-4bcb-a534-7f40b6c9e36f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:50.128529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe64e4-feaf-4bcb-a534-7f40b6c9e36f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:50.142126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 616/1238] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:50.148083 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e7d67-d2b2-4f96-a41a-d52ac09ab141 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:50.256204 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e7d67-d2b2-4f96-a41a-d52ac09ab141 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:50.291707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad7065-2627-4595-b788-8389d6ddc705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:50.292445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad7065-2627-4595-b788-8389d6ddc705 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:50.292751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad7065-2627-4595-b788-8389d6ddc705 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:50.293049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad7065-2627-4595-b788-8389d6ddc705 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:50.296518 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:50.302770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad7065-2627-4595-b788-8389d6ddc705 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:50.304370 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad7065-2627-4595-b788-8389d6ddc705 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-88ad7065-2627-4595-b788-8389d6ddc705 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:18:50.317554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:50.323638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:50.324848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:50.324848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:50.324848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:18:50.324848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:18:50.324848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:18:50.324848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:18:50.324848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:18:50.324848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:18:50.324848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 04:18:50.324848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:18:50.324848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 623/1239] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:50.325682 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe64e4-feaf-4bcb-a534-7f40b6c9e36f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:50.436174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe64e4-feaf-4bcb-a534-7f40b6c9e36f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:50.469482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecb369-1348-4167-bef0-3f9c46b30311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:50.470089 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecb369-1348-4167-bef0-3f9c46b30311 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:50.470397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecb369-1348-4167-bef0-3f9c46b30311 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:50.470685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecb369-1348-4167-bef0-3f9c46b30311 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:50.471410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecb369-1348-4167-bef0-3f9c46b30311 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:50.471722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecb369-1348-4167-bef0-3f9c46b30311 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:50.473183 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecb369-1348-4167-bef0-3f9c46b30311 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:50.492204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 617/1240] 10.222.0.60 () {56 vars in 1143 bytes} [Sat Sep 17 04:18:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:50.498430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad7065-2627-4595-b788-8389d6ddc705 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:50.606631 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad7065-2627-4595-b788-8389d6ddc705 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:50.646883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07484f16-7255-4dc6-9897-ce557a8386dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:50.647602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07484f16-7255-4dc6-9897-ce557a8386dc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:50.647916 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07484f16-7255-4dc6-9897-ce557a8386dc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:50.648214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07484f16-7255-4dc6-9897-ce557a8386dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:50.715305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:50.723034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07484f16-7255-4dc6-9897-ce557a8386dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:50.724703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07484f16-7255-4dc6-9897-ce557a8386dc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:50.837808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 624/1241] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:18:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:18:50.843906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecb369-1348-4167-bef0-3f9c46b30311 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:50.875435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:50.956580 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecb369-1348-4167-bef0-3f9c46b30311 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:50.994137 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04a51c-08a2-4ccf-85a4-4b916e278f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:50.996667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04a51c-08a2-4ccf-85a4-4b916e278f51 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:50.997004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04a51c-08a2-4ccf-85a4-4b916e278f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:50.997305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04a51c-08a2-4ccf-85a4-4b916e278f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:51.010408 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04a51c-08a2-4ccf-85a4-4b916e278f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:51.011867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04a51c-08a2-4ccf-85a4-4b916e278f51 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-fe04a51c-08a2-4ccf-85a4-4b916e278f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:18:51.028055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:51.029395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:18:51.030583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:18:51.030583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:51.030583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:18:51.030583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 618/1242] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:51.036110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07484f16-7255-4dc6-9897-ce557a8386dc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:51.137671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:51.143962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07484f16-7255-4dc6-9897-ce557a8386dc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:51.179556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffd049-9752-4e50-b733-1375c9a58a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:51.180277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffd049-9752-4e50-b733-1375c9a58a88 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:51.180596 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffd049-9752-4e50-b733-1375c9a58a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:51.180869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffd049-9752-4e50-b733-1375c9a58a88 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:51.181575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffd049-9752-4e50-b733-1375c9a58a88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:51.181892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffd049-9752-4e50-b733-1375c9a58a88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:51.183258 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffd049-9752-4e50-b733-1375c9a58a88 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:51.195026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 625/1243] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:18:51.200869 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04a51c-08a2-4ccf-85a4-4b916e278f51 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:51.284574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:51.309881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04a51c-08a2-4ccf-85a4-4b916e278f51 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:51.348190 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0e60c-4fd9-4d90-8881-5affe6438dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:51.348994 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0e60c-4fd9-4d90-8881-5affe6438dac None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:51.349486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0e60c-4fd9-4d90-8881-5affe6438dac None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:51.349689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0e60c-4fd9-4d90-8881-5affe6438dac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:51.356818 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0e60c-4fd9-4d90-8881-5affe6438dac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:51.358327 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0e60c-4fd9-4d90-8881-5affe6438dac None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-97a0e60c-4fd9-4d90-8881-5affe6438dac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:18:51.372396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:51.373673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:51.374765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:18:51.374765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:18:51.374765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:18:51.374765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:18:51.374765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:18:51.374765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:18:51.374765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:18:51.374765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:18:51.374765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 619/1244] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:18:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:51.381037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffd049-9752-4e50-b733-1375c9a58a88 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:51.490357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffd049-9752-4e50-b733-1375c9a58a88 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:51.524346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba34b4-6eb6-4981-9180-626194004b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:51.524982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba34b4-6eb6-4981-9180-626194004b76 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:51.525291 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba34b4-6eb6-4981-9180-626194004b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:51.525559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba34b4-6eb6-4981-9180-626194004b76 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:51.526280 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba34b4-6eb6-4981-9180-626194004b76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:51.526576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba34b4-6eb6-4981-9180-626194004b76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:51.528014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba34b4-6eb6-4981-9180-626194004b76 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:51.540638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 626/1245] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:18:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:51.546854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0e60c-4fd9-4d90-8881-5affe6438dac None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:51.654718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0e60c-4fd9-4d90-8881-5affe6438dac None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:51.703753 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692250c-5bc0-4eb2-97bf-35d7e3ec4de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:51.704693 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692250c-5bc0-4eb2-97bf-35d7e3ec4de0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:51.705123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692250c-5bc0-4eb2-97bf-35d7e3ec4de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:51.705573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692250c-5bc0-4eb2-97bf-35d7e3ec4de0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:51.706405 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692250c-5bc0-4eb2-97bf-35d7e3ec4de0 None admin] RBAC: Adding query filter params (role.id=3eccbe0521ef44d88574b9acd9f988db, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=e6d3b7887c13466ab8e3d9737f907c56) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:51.706815 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692250c-5bc0-4eb2-97bf-35d7e3ec4de0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:51.708710 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692250c-5bc0-4eb2-97bf-35d7e3ec4de0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:51.716923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 620/1246] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:18:51 2022] GET /identity/v3/role_assignments?user.id=e6d3b7887c13466ab8e3d9737f907c56&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=3eccbe0521ef44d88574b9acd9f988db => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:54.093037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5f645-f74d-4d02-9a84-43bd2152d91f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:54.093730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5f645-f74d-4d02-9a84-43bd2152d91f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:54.093730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5f645-f74d-4d02-9a84-43bd2152d91f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:54.095140 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 627/1247] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:18:54.101038 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf0659-cb8b-44da-9a7c-41ed22b0bc47 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:54.101599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf0659-cb8b-44da-9a7c-41ed22b0bc47 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:54.101599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf0659-cb8b-44da-9a7c-41ed22b0bc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:54.108011 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:54.121343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cbdf0659-cb8b-44da-9a7c-41ed22b0bc47 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:18:54.122530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cbdf0659-cb8b-44da-9a7c-41ed22b0bc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:18:54.192528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 621/1248] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:18:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:18:54.459066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5f645-f74d-4d02-9a84-43bd2152d91f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:54.478385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:54.576469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5f645-f74d-4d02-9a84-43bd2152d91f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:54.617148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00cc8e-e638-4369-beb6-3c9fd8b7bdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:54.617781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00cc8e-e638-4369-beb6-3c9fd8b7bdb1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:54.618053 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00cc8e-e638-4369-beb6-3c9fd8b7bdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:54.618318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00cc8e-e638-4369-beb6-3c9fd8b7bdb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:54.631368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00cc8e-e638-4369-beb6-3c9fd8b7bdb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ae00cc8e-e638-4369-beb6-3c9fd8b7bdb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:18:54.636216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:54.637530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:18:54.638495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 628/1249] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:18:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:54.644597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf0659-cb8b-44da-9a7c-41ed22b0bc47 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:54.757567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf0659-cb8b-44da-9a7c-41ed22b0bc47 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:54.789378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:54.794735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebde2fa-444c-4e9f-948e-5433fcc8149d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:54.795427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebde2fa-444c-4e9f-948e-5433fcc8149d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:54.795729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebde2fa-444c-4e9f-948e-5433fcc8149d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:54.796036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebde2fa-444c-4e9f-948e-5433fcc8149d None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:54.796784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebde2fa-444c-4e9f-948e-5433fcc8149d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:54.797143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebde2fa-444c-4e9f-948e-5433fcc8149d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:54.798560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebde2fa-444c-4e9f-948e-5433fcc8149d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:54.809892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 622/1250] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:18:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:18:54.815977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00cc8e-e638-4369-beb6-3c9fd8b7bdb1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:54.896457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:54.921663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00cc8e-e638-4369-beb6-3c9fd8b7bdb1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:54.959531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a032eff-5135-4e62-9e21-a4509e454bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:54.960173 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a032eff-5135-4e62-9e21-a4509e454bec None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:54.960558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a032eff-5135-4e62-9e21-a4509e454bec None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:54.960830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a032eff-5135-4e62-9e21-a4509e454bec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:54.974758 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a032eff-5135-4e62-9e21-a4509e454bec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:54.976294 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a032eff-5135-4e62-9e21-a4509e454bec None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8a032eff-5135-4e62-9e21-a4509e454bec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:18:54.991725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:54.992993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:18:54.994489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:18:54.994489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:54.994489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:18:54.994489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 629/1251] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:54.998980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebde2fa-444c-4e9f-948e-5433fcc8149d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:55.099671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebde2fa-444c-4e9f-948e-5433fcc8149d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:55.134557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efe501-c47b-48b5-947a-23dfeb0f8781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:55.135333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efe501-c47b-48b5-947a-23dfeb0f8781 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:55.135533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efe501-c47b-48b5-947a-23dfeb0f8781 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:55.135843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efe501-c47b-48b5-947a-23dfeb0f8781 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:55.136572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efe501-c47b-48b5-947a-23dfeb0f8781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:55.136952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efe501-c47b-48b5-947a-23dfeb0f8781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:55.138719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efe501-c47b-48b5-947a-23dfeb0f8781 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:55.151627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 623/1252] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:55.157654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a032eff-5135-4e62-9e21-a4509e454bec None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:55.265016 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a032eff-5135-4e62-9e21-a4509e454bec None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:55.292987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:55.301578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6391aa-07e1-49c3-814e-1af76e768051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:55.302215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6391aa-07e1-49c3-814e-1af76e768051 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:55.302508 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6391aa-07e1-49c3-814e-1af76e768051 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:55.302766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6391aa-07e1-49c3-814e-1af76e768051 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:55.311231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6391aa-07e1-49c3-814e-1af76e768051 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:55.312788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6391aa-07e1-49c3-814e-1af76e768051 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-9f6391aa-07e1-49c3-814e-1af76e768051 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:18:55.324477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:55.326186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:55.327483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:55.327483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:55.327483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:18:55.327483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:18:55.327483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:18:55.327483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:18:55.327483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:18:55.327483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:18:55.327483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 04:18:55.327483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:18:55.327483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 630/1253] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:18:55.332149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efe501-c47b-48b5-947a-23dfeb0f8781 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:55.399118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:55.446928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efe501-c47b-48b5-947a-23dfeb0f8781 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:55.481285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715169f9-e784-42ec-b262-6e2485131990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:55.481902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715169f9-e784-42ec-b262-6e2485131990 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:55.482281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715169f9-e784-42ec-b262-6e2485131990 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:55.482546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715169f9-e784-42ec-b262-6e2485131990 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:55.483280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715169f9-e784-42ec-b262-6e2485131990 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:55.483577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715169f9-e784-42ec-b262-6e2485131990 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:55.484985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715169f9-e784-42ec-b262-6e2485131990 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:55.502628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 624/1254] 10.222.0.60 () {56 vars in 1143 bytes} [Sat Sep 17 04:18:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:18:55.508651 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6391aa-07e1-49c3-814e-1af76e768051 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:55.618543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6391aa-07e1-49c3-814e-1af76e768051 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:55.655922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ea73a-f4c1-4f50-bbd8-fe10841ebbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:55.656511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ea73a-f4c1-4f50-bbd8-fe10841ebbdc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:55.656843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ea73a-f4c1-4f50-bbd8-fe10841ebbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:55.657219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ea73a-f4c1-4f50-bbd8-fe10841ebbdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:55.717814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:55.732169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ea73a-f4c1-4f50-bbd8-fe10841ebbdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:55.733833 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ea73a-f4c1-4f50-bbd8-fe10841ebbdc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:55.856392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 631/1255] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:18:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:18:55.857591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715169f9-e784-42ec-b262-6e2485131990 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:55.971387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715169f9-e784-42ec-b262-6e2485131990 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:55.986637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:56.011404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc59e3-609d-413d-a796-0c0c5c9950de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:56.012351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc59e3-609d-413d-a796-0c0c5c9950de None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:56.012617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc59e3-609d-413d-a796-0c0c5c9950de None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:56.012880 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc59e3-609d-413d-a796-0c0c5c9950de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:56.024227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc59e3-609d-413d-a796-0c0c5c9950de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:56.025733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc59e3-609d-413d-a796-0c0c5c9950de None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-67dc59e3-609d-413d-a796-0c0c5c9950de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:18:56.041833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:56.043069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:18:56.043973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:18:56.043973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:56.043973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:18:56.043973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 625/1256] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:56.049026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ea73a-f4c1-4f50-bbd8-fe10841ebbdc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:56.142167 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:56.155566 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ea73a-f4c1-4f50-bbd8-fe10841ebbdc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:56.194079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee48d0b-c06c-4b8d-9536-ba06d0c1520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:56.194876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee48d0b-c06c-4b8d-9536-ba06d0c1520e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:56.195116 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee48d0b-c06c-4b8d-9536-ba06d0c1520e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:56.195379 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee48d0b-c06c-4b8d-9536-ba06d0c1520e None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:56.196037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee48d0b-c06c-4b8d-9536-ba06d0c1520e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:56.196329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee48d0b-c06c-4b8d-9536-ba06d0c1520e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:56.197761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee48d0b-c06c-4b8d-9536-ba06d0c1520e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:56.210713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 632/1257] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:56.218392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc59e3-609d-413d-a796-0c0c5c9950de None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:56.323049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc59e3-609d-413d-a796-0c0c5c9950de None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:56.363382 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb8ce1-6522-424a-ab31-eb89b4702d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:56.364068 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb8ce1-6522-424a-ab31-eb89b4702d19 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:56.364365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb8ce1-6522-424a-ab31-eb89b4702d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:56.364662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb8ce1-6522-424a-ab31-eb89b4702d19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:56.372964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb8ce1-6522-424a-ab31-eb89b4702d19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:56.374457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb8ce1-6522-424a-ab31-eb89b4702d19 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-55fb8ce1-6522-424a-ab31-eb89b4702d19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:18:56.387423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:56.388395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:56.389348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:18:56.389348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:18:56.389348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:18:56.389348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:18:56.389348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:18:56.389348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:18:56.389348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:18:56.389348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:18:56.389348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 626/1258] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:18:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:56.394702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee48d0b-c06c-4b8d-9536-ba06d0c1520e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:56.495072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee48d0b-c06c-4b8d-9536-ba06d0c1520e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:56.530201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59367fa2-b094-4808-aa2c-3f0e06cbedc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:56.530812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59367fa2-b094-4808-aa2c-3f0e06cbedc9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:56.531088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59367fa2-b094-4808-aa2c-3f0e06cbedc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:56.531355 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59367fa2-b094-4808-aa2c-3f0e06cbedc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:56.532047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59367fa2-b094-4808-aa2c-3f0e06cbedc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:56.532349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59367fa2-b094-4808-aa2c-3f0e06cbedc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:56.533857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59367fa2-b094-4808-aa2c-3f0e06cbedc9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:56.545775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 633/1259] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:56.551963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb8ce1-6522-424a-ab31-eb89b4702d19 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:56.579428 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:56.656674 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb8ce1-6522-424a-ab31-eb89b4702d19 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:56.692468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b90b1a-d5dc-4c3b-b58b-5259a4b085b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:56.693061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b90b1a-d5dc-4c3b-b58b-5259a4b085b2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:56.693341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b90b1a-d5dc-4c3b-b58b-5259a4b085b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:56.693629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b90b1a-d5dc-4c3b-b58b-5259a4b085b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:56.694345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b90b1a-d5dc-4c3b-b58b-5259a4b085b2 None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=e6d3b7887c13466ab8e3d9737f907c56) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:56.694646 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b90b1a-d5dc-4c3b-b58b-5259a4b085b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:56.696105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b90b1a-d5dc-4c3b-b58b-5259a4b085b2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:56.701502 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 627/1260] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:18:56 2022] GET /identity/v3/role_assignments?user.id=e6d3b7887c13466ab8e3d9737f907c56&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:58.807618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e40e92-fef9-4374-bc27-6132732143aa None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:58.808218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e40e92-fef9-4374-bc27-6132732143aa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:58.808218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e40e92-fef9-4374-bc27-6132732143aa None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:58.809552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 634/1261] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:18:58.816103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72560d-a9f7-4fb6-a321-98dc0d23dc8f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:58.816829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72560d-a9f7-4fb6-a321-98dc0d23dc8f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:58.816999 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72560d-a9f7-4fb6-a321-98dc0d23dc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:58.843005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0b72560d-a9f7-4fb6-a321-98dc0d23dc8f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:18:58.844206 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0b72560d-a9f7-4fb6-a321-98dc0d23dc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:18:58.915218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 628/1262] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:18:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:18:59.178665 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e40e92-fef9-4374-bc27-6132732143aa None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:59.191152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:59.292891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e40e92-fef9-4374-bc27-6132732143aa None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:59.333035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c177f6-0d62-4e2d-80c3-04c57fe83381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:59.333862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c177f6-0d62-4e2d-80c3-04c57fe83381 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:59.334144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c177f6-0d62-4e2d-80c3-04c57fe83381 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:59.334460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c177f6-0d62-4e2d-80c3-04c57fe83381 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:59.348124 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c177f6-0d62-4e2d-80c3-04c57fe83381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b6c177f6-0d62-4e2d-80c3-04c57fe83381 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:18:59.354860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:59.355881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:18:59.357008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 635/1263] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:18:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:59.363569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72560d-a9f7-4fb6-a321-98dc0d23dc8f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:59.383835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:59.473835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72560d-a9f7-4fb6-a321-98dc0d23dc8f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:59.509528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef081f-4dc1-4d89-b245-9a855734f7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:59.510127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef081f-4dc1-4d89-b245-9a855734f7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:59.510404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef081f-4dc1-4d89-b245-9a855734f7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:59.510669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef081f-4dc1-4d89-b245-9a855734f7f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:59.511323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef081f-4dc1-4d89-b245-9a855734f7f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:59.511626 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef081f-4dc1-4d89-b245-9a855734f7f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:59.512979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef081f-4dc1-4d89-b245-9a855734f7f9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:59.521673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 629/1264] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:18:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:18:59.527747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c177f6-0d62-4e2d-80c3-04c57fe83381 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:59.596672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:59.630610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c177f6-0d62-4e2d-80c3-04c57fe83381 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:59.665879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97329cd6-f135-4a6f-a908-3a2d606eb2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:59.666505 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97329cd6-f135-4a6f-a908-3a2d606eb2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:59.666781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97329cd6-f135-4a6f-a908-3a2d606eb2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:59.667047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97329cd6-f135-4a6f-a908-3a2d606eb2e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:59.677447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97329cd6-f135-4a6f-a908-3a2d606eb2e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:59.678892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97329cd6-f135-4a6f-a908-3a2d606eb2e5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-97329cd6-f135-4a6f-a908-3a2d606eb2e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:18:59.693435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:18:59.694534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:18:59.695701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:18:59.695701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:18:59.695701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:18:59.695701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 636/1265] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:18:59.700624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef081f-4dc1-4d89-b245-9a855734f7f9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:59.782802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:18:59.804053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef081f-4dc1-4d89-b245-9a855734f7f9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:59.839738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfd937-3720-463e-b528-753f0d7000e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:18:59.840326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfd937-3720-463e-b528-753f0d7000e8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:18:59.840594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfd937-3720-463e-b528-753f0d7000e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:18:59.840855 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfd937-3720-463e-b528-753f0d7000e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:18:59.841519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfd937-3720-463e-b528-753f0d7000e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:18:59.841819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfd937-3720-463e-b528-753f0d7000e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:18:59.843175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfd937-3720-463e-b528-753f0d7000e8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:18:59.854766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 630/1266] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:18:59.863828 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97329cd6-f135-4a6f-a908-3a2d606eb2e5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:18:59.968426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97329cd6-f135-4a6f-a908-3a2d606eb2e5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:18:59.988994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:00.004177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc788dc-8e3e-42dd-9592-7604f1b6dd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:00.004786 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc788dc-8e3e-42dd-9592-7604f1b6dd66 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:00.005058 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc788dc-8e3e-42dd-9592-7604f1b6dd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:00.005360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc788dc-8e3e-42dd-9592-7604f1b6dd66 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:00.012794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc788dc-8e3e-42dd-9592-7604f1b6dd66 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:00.014262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc788dc-8e3e-42dd-9592-7604f1b6dd66 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6fc788dc-8e3e-42dd-9592-7604f1b6dd66 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:19:00.025412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:19:00.026570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:19:00.027525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:19:00.027525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:19:00.027525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:19:00.027525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:19:00.027525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:19:00.027525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:19:00.027525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:19:00.027525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:19:00.027525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 04:19:00.027525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:19:00.027525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 637/1267] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:18:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:19:00.032417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfd937-3720-463e-b528-753f0d7000e8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:00.132358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfd937-3720-463e-b528-753f0d7000e8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:00.169951 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a073f16-9b85-4196-ba26-89b3343aa5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:00.170544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a073f16-9b85-4196-ba26-89b3343aa5ac None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:00.170815 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a073f16-9b85-4196-ba26-89b3343aa5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:00.171076 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a073f16-9b85-4196-ba26-89b3343aa5ac None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:00.171789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a073f16-9b85-4196-ba26-89b3343aa5ac None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:00.172083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a073f16-9b85-4196-ba26-89b3343aa5ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:00.173526 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a073f16-9b85-4196-ba26-89b3343aa5ac None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:00.182104 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:00.191276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 631/1268] 10.222.0.60 () {56 vars in 1143 bytes} [Sat Sep 17 04:19:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:00.197511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc788dc-8e3e-42dd-9592-7604f1b6dd66 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:00.297289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc788dc-8e3e-42dd-9592-7604f1b6dd66 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:00.331257 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c7077-d2db-4078-9934-f521b48bf991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:00.331876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c7077-d2db-4078-9934-f521b48bf991 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:00.332150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c7077-d2db-4078-9934-f521b48bf991 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:00.332401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c7077-d2db-4078-9934-f521b48bf991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:00.343249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c7077-d2db-4078-9934-f521b48bf991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:00.344696 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c7077-d2db-4078-9934-f521b48bf991 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-559c7077-d2db-4078-9934-f521b48bf991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:19:00.358724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:19:00.359871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:19:00.360781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:19:00.360781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:19:00.360781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:19:00.360781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 638/1269] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 04:19:00.366087 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a073f16-9b85-4196-ba26-89b3343aa5ac None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:00.471298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a073f16-9b85-4196-ba26-89b3343aa5ac None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:00.507171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7396d48-2404-40f9-9bda-601ccb67e81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:00.507792 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7396d48-2404-40f9-9bda-601ccb67e81d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:00.508131 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7396d48-2404-40f9-9bda-601ccb67e81d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:00.508402 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7396d48-2404-40f9-9bda-601ccb67e81d None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:00.509041 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7396d48-2404-40f9-9bda-601ccb67e81d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:00.509389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7396d48-2404-40f9-9bda-601ccb67e81d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:00.510740 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7396d48-2404-40f9-9bda-601ccb67e81d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:00.521929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 632/1270] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:00.528516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c7077-d2db-4078-9934-f521b48bf991 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:00.566421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:00.642534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c7077-d2db-4078-9934-f521b48bf991 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:00.679657 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c08a4b-c12b-4db9-adae-5f977c1999f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:00.680304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c08a4b-c12b-4db9-adae-5f977c1999f3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:00.680581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c08a4b-c12b-4db9-adae-5f977c1999f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:00.680857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c08a4b-c12b-4db9-adae-5f977c1999f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:00.687901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c08a4b-c12b-4db9-adae-5f977c1999f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:00.689387 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c08a4b-c12b-4db9-adae-5f977c1999f3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-48c08a4b-c12b-4db9-adae-5f977c1999f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:19:00.700955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:19:00.701984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:19:00.703272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:19:00.703272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:19:00.703272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:19:00.703272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:19:00.703272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:19:00.703272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:19:00.703272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:19:00.703272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:19:00.703272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 639/1271] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:19:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:19:00.708250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7396d48-2404-40f9-9bda-601ccb67e81d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:00.772230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:00.814881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7396d48-2404-40f9-9bda-601ccb67e81d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:00.851205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a27bdc-ae3a-426f-a947-aa31f68f576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:00.851817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a27bdc-ae3a-426f-a947-aa31f68f576a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:00.852131 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a27bdc-ae3a-426f-a947-aa31f68f576a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:00.852400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a27bdc-ae3a-426f-a947-aa31f68f576a None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:00.853116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a27bdc-ae3a-426f-a947-aa31f68f576a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:00.853420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a27bdc-ae3a-426f-a947-aa31f68f576a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:00.854845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a27bdc-ae3a-426f-a947-aa31f68f576a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:00.867033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 633/1272] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:19:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:00.873277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c08a4b-c12b-4db9-adae-5f977c1999f3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:00.984592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:00.994870 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c08a4b-c12b-4db9-adae-5f977c1999f3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:01.030852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515c7df-5d86-4262-931f-90af8bda3fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:01.031440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515c7df-5d86-4262-931f-90af8bda3fda None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:01.031719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515c7df-5d86-4262-931f-90af8bda3fda None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:01.031986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515c7df-5d86-4262-931f-90af8bda3fda None admin] PATH_INFO: `/v3/projects/58466195aca94b188d82c1b3a647e767/users/e6d3b7887c13466ab8e3d9737f907c56/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:01.034026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515c7df-5d86-4262-931f-90af8bda3fda None admin] RBAC: Authorizing `identity:create_grant(project_id=58466195aca94b188d82c1b3a647e767, user_id=e6d3b7887c13466ab8e3d9737f907c56, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:01.035468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515c7df-5d86-4262-931f-90af8bda3fda None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:01.051453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 640/1273] 10.222.0.60 () {58 vars in 1439 bytes} [Sat Sep 17 04:19:00 2022] PUT /identity/v3/projects/58466195aca94b188d82c1b3a647e767/users/e6d3b7887c13466ab8e3d9737f907c56/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:19:03.144749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9200-6178-427d-bcb0-f20628679990 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:03.145685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9200-6178-427d-bcb0-f20628679990 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:03.145685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9200-6178-427d-bcb0-f20628679990 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:03.147855 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 634/1274] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:19:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:19:03.154124 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dbdf3-a695-4268-80f3-0b895fbceef1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:03.154572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dbdf3-a695-4268-80f3-0b895fbceef1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:03.154666 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dbdf3-a695-4268-80f3-0b895fbceef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:03.182564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-780dbdf3-a695-4268-80f3-0b895fbceef1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:19:03.183748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-780dbdf3-a695-4268-80f3-0b895fbceef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:19:03.251641 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 641/1275] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:19:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:19:03.527148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c9200-6178-427d-bcb0-f20628679990 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:03.643062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c9200-6178-427d-bcb0-f20628679990 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:03.660493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:03.678634 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679f38f-d12d-497f-a2d4-4257b9e06d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:03.679263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679f38f-d12d-497f-a2d4-4257b9e06d6e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:03.679541 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679f38f-d12d-497f-a2d4-4257b9e06d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:03.679802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679f38f-d12d-497f-a2d4-4257b9e06d6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:03.689849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679f38f-d12d-497f-a2d4-4257b9e06d6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-2679f38f-d12d-497f-a2d4-4257b9e06d6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 04:19:03.694571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:19:03.695676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:19:03.696567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 635/1276] 10.222.0.60 () {56 vars in 1096 bytes} [Sat Sep 17 04:19:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:19:03.702182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dbdf3-a695-4268-80f3-0b895fbceef1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:03.785112 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:03.812088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dbdf3-a695-4268-80f3-0b895fbceef1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:03.845967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba90d4a-1d35-48f8-9947-1f13b07d4cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:03.846562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba90d4a-1d35-48f8-9947-1f13b07d4cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:03.846837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba90d4a-1d35-48f8-9947-1f13b07d4cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:03.847103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba90d4a-1d35-48f8-9947-1f13b07d4cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:03.847764 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba90d4a-1d35-48f8-9947-1f13b07d4cb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:03.848056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba90d4a-1d35-48f8-9947-1f13b07d4cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:03.849460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba90d4a-1d35-48f8-9947-1f13b07d4cb0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:03.857604 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 642/1277] 10.222.0.60 () {56 vars in 1099 bytes} [Sat Sep 17 04:19:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:03.864819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679f38f-d12d-497f-a2d4-4257b9e06d6e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:03.973870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679f38f-d12d-497f-a2d4-4257b9e06d6e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:04.010244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ba535-7182-485d-b886-9cd3e226a4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:04.010856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ba535-7182-485d-b886-9cd3e226a4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:04.011140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ba535-7182-485d-b886-9cd3e226a4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:04.011403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ba535-7182-485d-b886-9cd3e226a4d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:04.021729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ba535-7182-485d-b886-9cd3e226a4d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:04.023159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ba535-7182-485d-b886-9cd3e226a4d5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-870ba535-7182-485d-b886-9cd3e226a4d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:19:04.037823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:19:04.038961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:19:04.040402 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:19:04.040402 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:19:04.040402 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:19:04.040402 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 636/1278] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:19:04.046205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba90d4a-1d35-48f8-9947-1f13b07d4cb0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:04.145022 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba90d4a-1d35-48f8-9947-1f13b07d4cb0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:04.180019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60975fa2-5ba2-4b24-bd06-b8aa89f6a5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:04.180610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60975fa2-5ba2-4b24-bd06-b8aa89f6a5ec None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:04.180861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60975fa2-5ba2-4b24-bd06-b8aa89f6a5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:04.181129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60975fa2-5ba2-4b24-bd06-b8aa89f6a5ec None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:04.181793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60975fa2-5ba2-4b24-bd06-b8aa89f6a5ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:04.182089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60975fa2-5ba2-4b24-bd06-b8aa89f6a5ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:04.183447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60975fa2-5ba2-4b24-bd06-b8aa89f6a5ec None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:04.195125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 643/1279] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:04.200978 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ba535-7182-485d-b886-9cd3e226a4d5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:04.237260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:04.318209 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ba535-7182-485d-b886-9cd3e226a4d5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:04.363420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ed0e7-531f-45bc-9532-7c67d6388a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:04.364038 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ed0e7-531f-45bc-9532-7c67d6388a3d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:04.364315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ed0e7-531f-45bc-9532-7c67d6388a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:04.364621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ed0e7-531f-45bc-9532-7c67d6388a3d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:04.372238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ed0e7-531f-45bc-9532-7c67d6388a3d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:04.373715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ed0e7-531f-45bc-9532-7c67d6388a3d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-3c7ed0e7-531f-45bc-9532-7c67d6388a3d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 04:19:04.385060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:19:04.386271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:19:04.387404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:19:04.387404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:19:04.387404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 04:19:04.387404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 04:19:04.387404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 04:19:04.387404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 04:19:04.387404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 04:19:04.387404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 04:19:04.387404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 04:19:04.387404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:19:04.387404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 637/1280] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:19:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:19:04.392266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60975fa2-5ba2-4b24-bd06-b8aa89f6a5ec None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:04.396393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:04.502515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60975fa2-5ba2-4b24-bd06-b8aa89f6a5ec None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:04.544409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac11797-3d40-4362-96d8-0bbdb449d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:04.545006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac11797-3d40-4362-96d8-0bbdb449d61f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:04.545318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac11797-3d40-4362-96d8-0bbdb449d61f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:04.545585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac11797-3d40-4362-96d8-0bbdb449d61f None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:04.546312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac11797-3d40-4362-96d8-0bbdb449d61f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:04.546609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac11797-3d40-4362-96d8-0bbdb449d61f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:04.548031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac11797-3d40-4362-96d8-0bbdb449d61f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:04.565364 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 644/1281] 10.222.0.60 () {56 vars in 1143 bytes} [Sat Sep 17 04:19:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:04.571408 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ed0e7-531f-45bc-9532-7c67d6388a3d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:04.666750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:04.681607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ed0e7-531f-45bc-9532-7c67d6388a3d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:04.717886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ec5e1-f435-468d-befe-d3b803276ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:04.718480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ec5e1-f435-468d-befe-d3b803276ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:04.718751 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ec5e1-f435-468d-befe-d3b803276ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:04.719015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ec5e1-f435-468d-befe-d3b803276ea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:04.788127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ec5e1-f435-468d-befe-d3b803276ea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:04.789662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ec5e1-f435-468d-befe-d3b803276ea5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:04.879343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:04.891805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 638/1282] 10.222.0.60 () {58 vars in 1282 bytes} [Sat Sep 17 04:19:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 04:19:04.898545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac11797-3d40-4362-96d8-0bbdb449d61f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:04.964837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:05.004746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac11797-3d40-4362-96d8-0bbdb449d61f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:05.039898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e83def-52bc-4318-8455-0a58d8465c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:05.040516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e83def-52bc-4318-8455-0a58d8465c73 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:05.040792 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e83def-52bc-4318-8455-0a58d8465c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:05.041056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e83def-52bc-4318-8455-0a58d8465c73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:05.051773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e83def-52bc-4318-8455-0a58d8465c73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:05.053244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e83def-52bc-4318-8455-0a58d8465c73 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b6e83def-52bc-4318-8455-0a58d8465c73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 04:19:05.068146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:19:05.069259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 04:19:05.070245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 04:19:05.070245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 04:19:05.070245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:19:05.070245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 645/1283] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:19:05.075198 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ec5e1-f435-468d-befe-d3b803276ea5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:05.171136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ec5e1-f435-468d-befe-d3b803276ea5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:05.205615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd7cb4-7213-47b0-8f39-543906566cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:05.206203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd7cb4-7213-47b0-8f39-543906566cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:05.206455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd7cb4-7213-47b0-8f39-543906566cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:05.206714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd7cb4-7213-47b0-8f39-543906566cf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:05.207335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd7cb4-7213-47b0-8f39-543906566cf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:05.207608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd7cb4-7213-47b0-8f39-543906566cf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:05.208973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd7cb4-7213-47b0-8f39-543906566cf2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:05.220233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 639/1284] 10.222.0.60 () {56 vars in 1109 bytes} [Sat Sep 17 04:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:05.226147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e83def-52bc-4318-8455-0a58d8465c73 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:05.328201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e83def-52bc-4318-8455-0a58d8465c73 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:05.342619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:05.362578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e35109-c108-44d1-a3a5-ab89d927acac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:05.363547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e35109-c108-44d1-a3a5-ab89d927acac None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:05.363815 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e35109-c108-44d1-a3a5-ab89d927acac None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:05.364068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e35109-c108-44d1-a3a5-ab89d927acac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:05.371632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e35109-c108-44d1-a3a5-ab89d927acac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:05.373053 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e35109-c108-44d1-a3a5-ab89d927acac None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-74e35109-c108-44d1-a3a5-ab89d927acac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 04:19:05.384176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:19:05.385183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:19:05.386162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 04:19:05.386162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 04:19:05.386162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 04:19:05.386162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 04:19:05.386162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 04:19:05.386162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 04:19:05.386162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 04:19:05.386162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 17 04:19:05.386162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 646/1285] 10.222.0.60 () {56 vars in 1111 bytes} [Sat Sep 17 04:19:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:19:05.391596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd7cb4-7213-47b0-8f39-543906566cf2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:05.442962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:05.495802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd7cb4-7213-47b0-8f39-543906566cf2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:05.529840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fc12a-e026-4e93-bab8-4d6001bf134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:05.530426 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fc12a-e026-4e93-bab8-4d6001bf134c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:05.530694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fc12a-e026-4e93-bab8-4d6001bf134c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:05.530954 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fc12a-e026-4e93-bab8-4d6001bf134c None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:05.531634 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fc12a-e026-4e93-bab8-4d6001bf134c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:05.531931 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fc12a-e026-4e93-bab8-4d6001bf134c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:05.533358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fc12a-e026-4e93-bab8-4d6001bf134c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:05.545735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 640/1286] 10.222.0.60 () {56 vars in 1148 bytes} [Sat Sep 17 04:19:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:19:05.552570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e35109-c108-44d1-a3a5-ab89d927acac None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:05.652523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e35109-c108-44d1-a3a5-ab89d927acac None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:05.689072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f45171-49be-4d97-98bc-3daa60a736ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:05.689692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f45171-49be-4d97-98bc-3daa60a736ee None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:05.689966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f45171-49be-4d97-98bc-3daa60a736ee None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:05.690234 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f45171-49be-4d97-98bc-3daa60a736ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:05.690948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f45171-49be-4d97-98bc-3daa60a736ee None admin] RBAC: Adding query filter params (role.id=cbe495373d2646beae215fdffe6f2ce9, scope.project.id=58466195aca94b188d82c1b3a647e767, user.id=e6d3b7887c13466ab8e3d9737f907c56) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:05.691248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f45171-49be-4d97-98bc-3daa60a736ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:05.692673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f45171-49be-4d97-98bc-3daa60a736ee None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:05.698718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 647/1287] 10.222.0.60 () {56 vars in 1374 bytes} [Sat Sep 17 04:19:05 2022] GET /identity/v3/role_assignments?user.id=e6d3b7887c13466ab8e3d9737f907c56&scope.project.id=58466195aca94b188d82c1b3a647e767&role.id=cbe495373d2646beae215fdffe6f2ce9 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:07.847468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98769fe9-9c89-49a6-96af-d7f67ffc4b22 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:07.848079 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98769fe9-9c89-49a6-96af-d7f67ffc4b22 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:07.848079 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98769fe9-9c89-49a6-96af-d7f67ffc4b22 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:07.849372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 641/1288] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:19:07.854897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff160b2d-b55a-42ec-90d8-497423696452 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:07.855390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff160b2d-b55a-42ec-90d8-497423696452 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:07.855390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff160b2d-b55a-42ec-90d8-497423696452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:07.875341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ff160b2d-b55a-42ec-90d8-497423696452 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:19:07.876533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ff160b2d-b55a-42ec-90d8-497423696452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:19:07.907249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:07.950143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 648/1289] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:19:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:19:08.225151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98769fe9-9c89-49a6-96af-d7f67ffc4b22 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:08.342005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98769fe9-9c89-49a6-96af-d7f67ffc4b22 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:08.357971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:08.390719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cc831-fa68-4f4e-aff5-66386e4fea82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:08.391496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cc831-fa68-4f4e-aff5-66386e4fea82 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:08.391852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cc831-fa68-4f4e-aff5-66386e4fea82 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:08.392151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cc831-fa68-4f4e-aff5-66386e4fea82 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:08.392945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cc831-fa68-4f4e-aff5-66386e4fea82 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:08.394627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cc831-fa68-4f4e-aff5-66386e4fea82 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-330cc831-fa68-4f4e-aff5-66386e4fea82 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:19:08.405061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:19:08.407029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:19:08.408180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 642/1290] 10.222.0.60 () {56 vars in 1117 bytes} [Sat Sep 17 04:19:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:19:08.412713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff160b2d-b55a-42ec-90d8-497423696452 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:08.524027 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff160b2d-b55a-42ec-90d8-497423696452 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:08.560396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584bcd1-6443-4ab9-abc2-5f22a7de69da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:08.560996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584bcd1-6443-4ab9-abc2-5f22a7de69da None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:08.561308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584bcd1-6443-4ab9-abc2-5f22a7de69da None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:08.561583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584bcd1-6443-4ab9-abc2-5f22a7de69da None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:08.562213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584bcd1-6443-4ab9-abc2-5f22a7de69da None admin] RBAC: Adding query filter params (name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:08.562518 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584bcd1-6443-4ab9-abc2-5f22a7de69da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:08.563890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584bcd1-6443-4ab9-abc2-5f22a7de69da None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:08.568872 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 649/1291] 10.222.0.60 () {56 vars in 1116 bytes} [Sat Sep 17 04:19:08 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:08.574932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cc831-fa68-4f4e-aff5-66386e4fea82 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:08.675504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cc831-fa68-4f4e-aff5-66386e4fea82 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:08.710442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ed4bd-0a21-447f-b1b6-d41f921af1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:08.711070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ed4bd-0a21-447f-b1b6-d41f921af1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:08.711346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ed4bd-0a21-447f-b1b6-d41f921af1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:08.711616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ed4bd-0a21-447f-b1b6-d41f921af1f6 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:08.712252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ed4bd-0a21-447f-b1b6-d41f921af1f6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:08.712551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ed4bd-0a21-447f-b1b6-d41f921af1f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:08.714016 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ed4bd-0a21-447f-b1b6-d41f921af1f6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:08.718438 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 643/1292] 10.222.0.60 () {56 vars in 1116 bytes} [Sat Sep 17 04:19:08 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:10.835048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3b880-b9e7-4d80-bbe0-70cbd18a6abe None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:10.836028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3b880-b9e7-4d80-bbe0-70cbd18a6abe None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:10.836028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3b880-b9e7-4d80-bbe0-70cbd18a6abe None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:10.838197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 650/1293] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:19:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:19:10.843663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a0f7a-4d7c-40a6-8871-58cee1ac85f4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:10.844103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a0f7a-4d7c-40a6-8871-58cee1ac85f4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:10.844195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a0f7a-4d7c-40a6-8871-58cee1ac85f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:10.864433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d98a0f7a-4d7c-40a6-8871-58cee1ac85f4 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:19:10.865635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d98a0f7a-4d7c-40a6-8871-58cee1ac85f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:19:10.897064 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:10.943352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 644/1294] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:19:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:19:11.218338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3b880-b9e7-4d80-bbe0-70cbd18a6abe None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:11.253220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:11.325230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3b880-b9e7-4d80-bbe0-70cbd18a6abe None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:11.361530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c85c0-b724-4794-835f-38c3504535d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:11.362129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c85c0-b724-4794-835f-38c3504535d7 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:11.362407 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c85c0-b724-4794-835f-38c3504535d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:11.363039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c85c0-b724-4794-835f-38c3504535d7 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:11.363787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c85c0-b724-4794-835f-38c3504535d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:11.365182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c85c0-b724-4794-835f-38c3504535d7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:11.381721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 651/1295] 10.222.0.60 () {60 vars in 1141 bytes} [Sat Sep 17 04:19:11 2022] POST /identity/v3/services => generated 246 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:19:13.584121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12764e07-2b8a-4284-bc3c-fdfc514f2811 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:13.584859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12764e07-2b8a-4284-bc3c-fdfc514f2811 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:13.584859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12764e07-2b8a-4284-bc3c-fdfc514f2811 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:13.587060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 645/1296] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:19:13.594348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558eda95-5f1d-46a7-bef4-880e66e60fa3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:13.595046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558eda95-5f1d-46a7-bef4-880e66e60fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:13.595245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558eda95-5f1d-46a7-bef4-880e66e60fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:13.623203 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-558eda95-5f1d-46a7-bef4-880e66e60fa3 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:19:13.624394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-558eda95-5f1d-46a7-bef4-880e66e60fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:19:13.699269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 652/1297] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:19:13 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:19:13.967467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12764e07-2b8a-4284-bc3c-fdfc514f2811 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:14.079941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12764e07-2b8a-4284-bc3c-fdfc514f2811 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:14.118607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52547c5-c7f1-4411-ba5b-4b9d77212cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:14.119227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52547c5-c7f1-4411-ba5b-4b9d77212cda None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:14.119504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52547c5-c7f1-4411-ba5b-4b9d77212cda None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:14.119772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52547c5-c7f1-4411-ba5b-4b9d77212cda None admin] PATH_INFO: `/v3/services/placement` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:14.120346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52547c5-c7f1-4411-ba5b-4b9d77212cda None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:14.121741 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52547c5-c7f1-4411-ba5b-4b9d77212cda None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-f52547c5-c7f1-4411-ba5b-4b9d77212cda None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:19:14.130862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:19:14.132121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:19:14.133365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 646/1298] 10.222.0.60 () {56 vars in 1117 bytes} [Sat Sep 17 04:19:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:19:14.137974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558eda95-5f1d-46a7-bef4-880e66e60fa3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:14.156403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:14.246329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558eda95-5f1d-46a7-bef4-880e66e60fa3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:14.280751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094ff8c-35db-43b5-90ee-05fd6ce0bdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:14.281373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094ff8c-35db-43b5-90ee-05fd6ce0bdb4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:14.281648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094ff8c-35db-43b5-90ee-05fd6ce0bdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:14.281915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094ff8c-35db-43b5-90ee-05fd6ce0bdb4 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:14.282539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094ff8c-35db-43b5-90ee-05fd6ce0bdb4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:14.282830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094ff8c-35db-43b5-90ee-05fd6ce0bdb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:14.284227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094ff8c-35db-43b5-90ee-05fd6ce0bdb4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:14.289410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 653/1299] 10.222.0.60 () {56 vars in 1116 bytes} [Sat Sep 17 04:19:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:14.299063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52547c5-c7f1-4411-ba5b-4b9d77212cda None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:14.334063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:14.404272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52547c5-c7f1-4411-ba5b-4b9d77212cda None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:14.440353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8c9d7-0dc2-414a-834d-81392dbe0ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:14.440958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8c9d7-0dc2-414a-834d-81392dbe0ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:14.441266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8c9d7-0dc2-414a-834d-81392dbe0ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:14.441535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8c9d7-0dc2-414a-834d-81392dbe0ba9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:14.442242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8c9d7-0dc2-414a-834d-81392dbe0ba9 None admin] RBAC: Adding query filter params (interface=public, service_id=0c8d9feca99e413aabe2e391ca8795d6, region_id=RegionOne) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:14.442550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8c9d7-0dc2-414a-834d-81392dbe0ba9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:14.443907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8c9d7-0dc2-414a-834d-81392dbe0ba9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:14.448906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 647/1300] 10.222.0.60 () {56 vars in 1251 bytes} [Sat Sep 17 04:19:14 2022] GET /identity/v3/endpoints?service_id=0c8d9feca99e413aabe2e391ca8795d6&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:14.454869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094ff8c-35db-43b5-90ee-05fd6ce0bdb4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:14.536515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:14.554887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094ff8c-35db-43b5-90ee-05fd6ce0bdb4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:14.589008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d00f8-2f47-49a7-a7dd-edf61c9b2440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:14.589621 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d00f8-2f47-49a7-a7dd-edf61c9b2440 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:14.589903 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d00f8-2f47-49a7-a7dd-edf61c9b2440 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:14.590155 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d00f8-2f47-49a7-a7dd-edf61c9b2440 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:14.590748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d00f8-2f47-49a7-a7dd-edf61c9b2440 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:14.592109 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d00f8-2f47-49a7-a7dd-edf61c9b2440 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:14.598951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 654/1301] 10.222.0.60 () {56 vars in 1087 bytes} [Sat Sep 17 04:19:14 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:19:16.754037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c10bac-0dd0-4684-9e9e-461dfb74365c None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:16.754559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c10bac-0dd0-4684-9e9e-461dfb74365c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:16.754559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c10bac-0dd0-4684-9e9e-461dfb74365c None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:16.755929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 648/1302] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:19:16.762401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c577c4c-e262-4a21-8cc3-4501a8a88fee None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:16.763156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c577c4c-e262-4a21-8cc3-4501a8a88fee None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:16.763342 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c577c4c-e262-4a21-8cc3-4501a8a88fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:16.784570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0c577c4c-e262-4a21-8cc3-4501a8a88fee None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:19:16.785784 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0c577c4c-e262-4a21-8cc3-4501a8a88fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:19:16.854950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 655/1303] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:19:16 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 04:19:17.116714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c10bac-0dd0-4684-9e9e-461dfb74365c None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:17.216338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:17.226361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c10bac-0dd0-4684-9e9e-461dfb74365c None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:17.263685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c322-c4bf-42b4-b768-8ab36379d79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:17.264302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c322-c4bf-42b4-b768-8ab36379d79f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:17.264576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c322-c4bf-42b4-b768-8ab36379d79f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:17.264812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c322-c4bf-42b4-b768-8ab36379d79f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:17.265445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c322-c4bf-42b4-b768-8ab36379d79f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:17.266810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c322-c4bf-42b4-b768-8ab36379d79f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-8c83c322-c4bf-42b4-b768-8ab36379d79f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 04:19:17.276549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 04:19:17.277907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 17 04:19:17.279015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 649/1304] 10.222.0.60 () {56 vars in 1117 bytes} [Sat Sep 17 04:19:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 04:19:17.284520 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c577c4c-e262-4a21-8cc3-4501a8a88fee None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:17.371601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:17.398629 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c577c4c-e262-4a21-8cc3-4501a8a88fee None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:17.433583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848afd2f-06c7-4ad1-85d5-261ef56f9fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:17.434183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848afd2f-06c7-4ad1-85d5-261ef56f9fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:17.434446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848afd2f-06c7-4ad1-85d5-261ef56f9fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:17.434709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848afd2f-06c7-4ad1-85d5-261ef56f9fa0 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:17.435322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848afd2f-06c7-4ad1-85d5-261ef56f9fa0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:19:17.435618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848afd2f-06c7-4ad1-85d5-261ef56f9fa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:17.436968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848afd2f-06c7-4ad1-85d5-261ef56f9fa0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:17.442650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 656/1305] 10.222.0.60 () {56 vars in 1116 bytes} [Sat Sep 17 04:19:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:19:17.449491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c322-c4bf-42b4-b768-8ab36379d79f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:19:17.563966 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c322-c4bf-42b4-b768-8ab36379d79f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:19:17.598577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27aac36-4f73-49c3-961e-3e9475c1ec49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:19:17.599158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aac36-4f73-49c3-961e-3e9475c1ec49 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:19:17.599430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aac36-4f73-49c3-961e-3e9475c1ec49 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:19:17.599694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aac36-4f73-49c3-961e-3e9475c1ec49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:19:17.600351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27aac36-4f73-49c3-961e-3e9475c1ec49 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:19:17.601785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27aac36-4f73-49c3-961e-3e9475c1ec49 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:19:17.623450 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 650/1306] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:19:17 2022] POST /identity/v3/endpoints => generated 331 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:19:27.440303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=30 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:19:27.618961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:21.671889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039ded0-0914-4b2d-8fb0-175df37236e0 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:21.672392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039ded0-0914-4b2d-8fb0-175df37236e0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:21.672657 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039ded0-0914-4b2d-8fb0-175df37236e0 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:21.674189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 657/1307] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:21.679621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb163ce-7ba5-4e13-b541-c7a6d66e2bf6 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:21.680039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb163ce-7ba5-4e13-b541-c7a6d66e2bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:21.680223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb163ce-7ba5-4e13-b541-c7a6d66e2bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:21.823610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cdb163ce-7ba5-4e13-b541-c7a6d66e2bf6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:21.825166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cdb163ce-7ba5-4e13-b541-c7a6d66e2bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:21.977188 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 651/1308] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:20:22.268276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691bd1c-2d91-4c90-9bee-39019be9baf7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:22.268623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691bd1c-2d91-4c90-9bee-39019be9baf7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:22.268816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691bd1c-2d91-4c90-9bee-39019be9baf7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:22.270319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 658/1309] 10.222.0.60 () {54 vars in 963 bytes} [Sat Sep 17 04:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:22.276739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7e05d-331c-4e00-bfa2-706d84aa7f2a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:22.277062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7e05d-331c-4e00-bfa2-706d84aa7f2a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:22.277363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7e05d-331c-4e00-bfa2-706d84aa7f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:22.370353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-60f7e05d-331c-4e00-bfa2-706d84aa7f2a None None] MFA Rules not processed for user `c9360be7ee9e4739b20c59955d28da47`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:22.371558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-60f7e05d-331c-4e00-bfa2-706d84aa7f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:22.394276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:22.456876 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 652/1310] 10.222.0.60 () {58 vars in 1060 bytes} [Sat Sep 17 04:20:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:20:22.465793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691bd1c-2d91-4c90-9bee-39019be9baf7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:20:22.467852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2691bd1c-2d91-4c90-9bee-39019be9baf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:22.580285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691bd1c-2d91-4c90-9bee-39019be9baf7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:20:22.670558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c075a-abe6-48ab-a7b3-d75ea2d81db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:20:22.671439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c075a-abe6-48ab-a7b3-d75ea2d81db5 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:22.671912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c075a-abe6-48ab-a7b3-d75ea2d81db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:22.672767 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c075a-abe6-48ab-a7b3-d75ea2d81db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:22.674699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e61c075a-abe6-48ab-a7b3-d75ea2d81db5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:22.733819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:22.797007 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c075a-abe6-48ab-a7b3-d75ea2d81db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:20:22.798763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c075a-abe6-48ab-a7b3-d75ea2d81db5 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:20:22.818993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 659/1311] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:20:25.270227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a8c5-938f-4f6a-8f04-9ecbb0b3ff91 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:25.270760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a8c5-938f-4f6a-8f04-9ecbb0b3ff91 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:25.270760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a8c5-938f-4f6a-8f04-9ecbb0b3ff91 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:25.272110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 653/1312] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:25.278689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e833d2-189c-43a5-877e-e1a14d771bb4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:25.279434 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e833d2-189c-43a5-877e-e1a14d771bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:25.279562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e833d2-189c-43a5-877e-e1a14d771bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:25.362705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-50e833d2-189c-43a5-877e-e1a14d771bb4 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:25.363863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-50e833d2-189c-43a5-877e-e1a14d771bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:25.474712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 660/1313] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:20:25.758666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91a8c5-938f-4f6a-8f04-9ecbb0b3ff91 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:20:25.761043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8e91a8c5-938f-4f6a-8f04-9ecbb0b3ff91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:25.884933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91a8c5-938f-4f6a-8f04-9ecbb0b3ff91 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:20:25.963420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:25.965871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75ba41-d537-46ee-a6fb-2b6af1562709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:20:25.966701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75ba41-d537-46ee-a6fb-2b6af1562709 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:25.967177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75ba41-d537-46ee-a6fb-2b6af1562709 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:25.967698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75ba41-d537-46ee-a6fb-2b6af1562709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:25.969374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-be75ba41-d537-46ee-a6fb-2b6af1562709 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:26.081184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75ba41-d537-46ee-a6fb-2b6af1562709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:20:26.082613 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75ba41-d537-46ee-a6fb-2b6af1562709 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:20:26.101369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 654/1314] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:20:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:20:28.561163 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71889647-3e99-4fc2-9e78-56d994514e2e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:28.561761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71889647-3e99-4fc2-9e78-56d994514e2e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:28.561761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71889647-3e99-4fc2-9e78-56d994514e2e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:28.563752 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 661/1315] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:20:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:28.573140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83695ba1-d389-4abf-bf64-ff204d7d7f59 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:28.573526 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83695ba1-d389-4abf-bf64-ff204d7d7f59 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:28.573702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83695ba1-d389-4abf-bf64-ff204d7d7f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:28.646043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-83695ba1-d389-4abf-bf64-ff204d7d7f59 None None] MFA Rules not processed for user `f361cc03b1bf40ce81b648b897538d31`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:28.647259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-83695ba1-d389-4abf-bf64-ff204d7d7f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:28.700133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:28.743555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 655/1316] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:20:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:20:29.038456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71889647-3e99-4fc2-9e78-56d994514e2e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:20:29.040630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-71889647-3e99-4fc2-9e78-56d994514e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:29.051446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:29.148517 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71889647-3e99-4fc2-9e78-56d994514e2e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:20:29.226964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f3a29-59ff-4f55-a29a-65784dd0aa60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:20:29.227582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3a29-59ff-4f55-a29a-65784dd0aa60 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:29.227855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3a29-59ff-4f55-a29a-65784dd0aa60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:29.228121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3a29-59ff-4f55-a29a-65784dd0aa60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:29.229725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-688f3a29-59ff-4f55-a29a-65784dd0aa60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:29.297438 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:29.356947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f3a29-59ff-4f55-a29a-65784dd0aa60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:20:29.359161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f3a29-59ff-4f55-a29a-65784dd0aa60 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:20:29.380738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 662/1317] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:20:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:20:32.084379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcede80d-3e60-4532-a527-d45e39cb9fbf None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:32.085419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcede80d-3e60-4532-a527-d45e39cb9fbf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:32.085419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcede80d-3e60-4532-a527-d45e39cb9fbf None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:32.087438 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 656/1318] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:20:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:32.096559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e2922-559b-46b3-b6bb-04cf835d74f0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:32.096984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e2922-559b-46b3-b6bb-04cf835d74f0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:32.097149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e2922-559b-46b3-b6bb-04cf835d74f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:32.167624 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-712e2922-559b-46b3-b6bb-04cf835d74f0 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:32.168815 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-712e2922-559b-46b3-b6bb-04cf835d74f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:32.271429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 663/1319] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:20:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:20:32.554622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcede80d-3e60-4532-a527-d45e39cb9fbf None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:20:32.557043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fcede80d-3e60-4532-a527-d45e39cb9fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:32.672732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcede80d-3e60-4532-a527-d45e39cb9fbf None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:20:32.750724 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938519b8-bc8e-4cec-b7fe-e3f845fe2123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:20:32.751330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938519b8-bc8e-4cec-b7fe-e3f845fe2123 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:32.751610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938519b8-bc8e-4cec-b7fe-e3f845fe2123 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:32.751877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938519b8-bc8e-4cec-b7fe-e3f845fe2123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:32.753513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-938519b8-bc8e-4cec-b7fe-e3f845fe2123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:32.760622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:32.878470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938519b8-bc8e-4cec-b7fe-e3f845fe2123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:20:32.879811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938519b8-bc8e-4cec-b7fe-e3f845fe2123 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:20:32.898200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 657/1320] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:20:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:20:35.686091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd519c9-e206-41da-ad39-49592d1c2569 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:35.686619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd519c9-e206-41da-ad39-49592d1c2569 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:35.686619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd519c9-e206-41da-ad39-49592d1c2569 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:35.688092 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 664/1321] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:35.693479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c2a0d-77d7-49a0-a7c1-2def26592f0d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:35.693910 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c2a0d-77d7-49a0-a7c1-2def26592f0d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:35.694019 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c2a0d-77d7-49a0-a7c1-2def26592f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:35.768180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-8b1c2a0d-77d7-49a0-a7c1-2def26592f0d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:35.769443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8b1c2a0d-77d7-49a0-a7c1-2def26592f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:35.832486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:35.876196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 658/1322] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:20:36.167860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c2941-77b5-4ca9-8e61-1012c0d8edea None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:36.168120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c2941-77b5-4ca9-8e61-1012c0d8edea None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:36.168397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c2941-77b5-4ca9-8e61-1012c0d8edea None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:36.169758 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 665/1323] 10.222.0.60 () {54 vars in 963 bytes} [Sat Sep 17 04:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:36.176196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb507ca-2eef-446e-8f71-8af6f24109be None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:36.176556 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb507ca-2eef-446e-8f71-8af6f24109be None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:36.176891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb507ca-2eef-446e-8f71-8af6f24109be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:36.256623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7eb507ca-2eef-446e-8f71-8af6f24109be None None] MFA Rules not processed for user `c9360be7ee9e4739b20c59955d28da47`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:36.257854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7eb507ca-2eef-446e-8f71-8af6f24109be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:36.334583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 659/1324] 10.222.0.60 () {58 vars in 1060 bytes} [Sat Sep 17 04:20:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:20:36.344003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c2941-77b5-4ca9-8e61-1012c0d8edea None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:20:36.345659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3e7c2941-77b5-4ca9-8e61-1012c0d8edea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:36.365876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:36.452959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c2941-77b5-4ca9-8e61-1012c0d8edea None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:20:36.528319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a1045-f3fd-48a7-a3e7-e668d269b529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:20:36.528930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a1045-f3fd-48a7-a3e7-e668d269b529 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:36.529237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a1045-f3fd-48a7-a3e7-e668d269b529 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:36.529507 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a1045-f3fd-48a7-a3e7-e668d269b529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:36.531088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-568a1045-f3fd-48a7-a3e7-e668d269b529 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:36.603873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:36.642334 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a1045-f3fd-48a7-a3e7-e668d269b529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:20:36.643798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a1045-f3fd-48a7-a3e7-e668d269b529 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:20:36.663942 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 666/1325] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:20:39.085865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba5faa-4946-4c14-94d1-c0539158b2fd None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:39.086406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba5faa-4946-4c14-94d1-c0539158b2fd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:39.086406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba5faa-4946-4c14-94d1-c0539158b2fd None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:39.087700 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 660/1326] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:39.093285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d5b45-1a6d-48cf-be29-16ea4e0f7211 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:39.093760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d5b45-1a6d-48cf-be29-16ea4e0f7211 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:39.093857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d5b45-1a6d-48cf-be29-16ea4e0f7211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:39.160715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-921d5b45-1a6d-48cf-be29-16ea4e0f7211 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:39.161958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-921d5b45-1a6d-48cf-be29-16ea4e0f7211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:39.261499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 667/1327] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:20:39.546336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba5faa-4946-4c14-94d1-c0539158b2fd None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:20:39.547906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e6ba5faa-4946-4c14-94d1-c0539158b2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:39.614893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:39.669565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba5faa-4946-4c14-94d1-c0539158b2fd None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:20:39.748897 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c3e66-0d7c-4c1a-9d6a-f998ad2d32d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:20:39.749551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c3e66-0d7c-4c1a-9d6a-f998ad2d32d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:39.749827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c3e66-0d7c-4c1a-9d6a-f998ad2d32d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:39.750104 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c3e66-0d7c-4c1a-9d6a-f998ad2d32d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:39.751704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4f7c3e66-0d7c-4c1a-9d6a-f998ad2d32d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:39.858540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:39.860990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c3e66-0d7c-4c1a-9d6a-f998ad2d32d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:20:39.862735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c3e66-0d7c-4c1a-9d6a-f998ad2d32d7 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:20:39.883742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 661/1328] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:20:42.111201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf008d5-3a6e-47a8-a037-1cd1b8bf14ab None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:42.111798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf008d5-3a6e-47a8-a037-1cd1b8bf14ab None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:42.111798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf008d5-3a6e-47a8-a037-1cd1b8bf14ab None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:42.113129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 668/1329] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:42.118977 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a200d-1da8-4506-85a0-25d511d1af24 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:42.119482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a200d-1da8-4506-85a0-25d511d1af24 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:42.119482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a200d-1da8-4506-85a0-25d511d1af24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:42.191739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-679a200d-1da8-4506-85a0-25d511d1af24 None None] MFA Rules not processed for user `f361cc03b1bf40ce81b648b897538d31`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:42.192963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-679a200d-1da8-4506-85a0-25d511d1af24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:42.291615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 662/1330] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:20:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:20:42.569979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf008d5-3a6e-47a8-a037-1cd1b8bf14ab None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:20:42.571634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-daf008d5-3a6e-47a8-a037-1cd1b8bf14ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:42.604606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:42.680918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf008d5-3a6e-47a8-a037-1cd1b8bf14ab None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:20:42.756583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87cef42-f09c-4226-ae3e-f75f2fcb25fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:20:42.757288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87cef42-f09c-4226-ae3e-f75f2fcb25fd service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:42.757562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87cef42-f09c-4226-ae3e-f75f2fcb25fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:42.757826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87cef42-f09c-4226-ae3e-f75f2fcb25fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:42.759488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a87cef42-f09c-4226-ae3e-f75f2fcb25fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:42.843772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:42.877162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87cef42-f09c-4226-ae3e-f75f2fcb25fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:20:42.878639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87cef42-f09c-4226-ae3e-f75f2fcb25fd service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:20:42.897359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 669/1331] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:20:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:20:45.289829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e5b63-2a15-4eb3-a5a9-eb9ab0425de1 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:45.290394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e5b63-2a15-4eb3-a5a9-eb9ab0425de1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:45.290394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e5b63-2a15-4eb3-a5a9-eb9ab0425de1 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:45.291710 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 663/1332] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:20:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:45.297372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd73020-7c09-45d2-9441-53f42fc4172c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:45.297836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd73020-7c09-45d2-9441-53f42fc4172c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:45.297945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd73020-7c09-45d2-9441-53f42fc4172c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:45.364115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7cd73020-7c09-45d2-9441-53f42fc4172c None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:45.365356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7cd73020-7c09-45d2-9441-53f42fc4172c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:45.473666 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 670/1333] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:20:45.754498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e5b63-2a15-4eb3-a5a9-eb9ab0425de1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:20:45.756974 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3d8e5b63-2a15-4eb3-a5a9-eb9ab0425de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:45.844191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:45.880912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e5b63-2a15-4eb3-a5a9-eb9ab0425de1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:20:45.970625 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e3f2d-4dd5-435a-a562-8cae32a45caa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:20:45.971453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e3f2d-4dd5-435a-a562-8cae32a45caa service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:45.971783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e3f2d-4dd5-435a-a562-8cae32a45caa service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:45.972118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e3f2d-4dd5-435a-a562-8cae32a45caa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:45.974720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e67e3f2d-4dd5-435a-a562-8cae32a45caa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:46.098902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e3f2d-4dd5-435a-a562-8cae32a45caa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:20:46.100327 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e3f2d-4dd5-435a-a562-8cae32a45caa service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:20:46.110933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:46.120735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 664/1334] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:20:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:20:48.308160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8204b9-f4a0-489b-a58a-00f04c896a80 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:48.308735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8204b9-f4a0-489b-a58a-00f04c896a80 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:48.308735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8204b9-f4a0-489b-a58a-00f04c896a80 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:48.310077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 671/1335] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:48.316485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d19f98-1819-4de4-bcbf-8a83f4ad1b27 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:48.317278 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d19f98-1819-4de4-bcbf-8a83f4ad1b27 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:48.317461 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d19f98-1819-4de4-bcbf-8a83f4ad1b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:48.387578 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b9d19f98-1819-4de4-bcbf-8a83f4ad1b27 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:48.388758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b9d19f98-1819-4de4-bcbf-8a83f4ad1b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:48.493967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 665/1336] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:20:48.778308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8204b9-f4a0-489b-a58a-00f04c896a80 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:20:48.779847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5b8204b9-f4a0-489b-a58a-00f04c896a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:48.822710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:48.893062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8204b9-f4a0-489b-a58a-00f04c896a80 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:20:48.967445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d84de-7541-4b80-b4d9-7f65d686e488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:20:48.967965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d84de-7541-4b80-b4d9-7f65d686e488 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:48.968241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d84de-7541-4b80-b4d9-7f65d686e488 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:48.968505 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d84de-7541-4b80-b4d9-7f65d686e488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:48.970108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b51d84de-7541-4b80-b4d9-7f65d686e488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:49.056094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:49.082527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d84de-7541-4b80-b4d9-7f65d686e488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:20:49.083973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d84de-7541-4b80-b4d9-7f65d686e488 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:20:49.101856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 672/1337] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:20:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:20:51.591033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891da3f4-6adf-4f35-87be-6bf37fc8fc4a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:51.592128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891da3f4-6adf-4f35-87be-6bf37fc8fc4a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:51.592128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891da3f4-6adf-4f35-87be-6bf37fc8fc4a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:51.594106 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 666/1338] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:20:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:51.603292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43703b-d739-427f-b776-8db1a3ca8f60 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:51.603730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43703b-d739-427f-b776-8db1a3ca8f60 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:51.603832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43703b-d739-427f-b776-8db1a3ca8f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:51.672780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7a43703b-d739-427f-b776-8db1a3ca8f60 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:51.674109 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7a43703b-d739-427f-b776-8db1a3ca8f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:51.772202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 673/1339] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:20:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:20:52.061700 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891da3f4-6adf-4f35-87be-6bf37fc8fc4a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:20:52.064189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-891da3f4-6adf-4f35-87be-6bf37fc8fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:52.145630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:52.175176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891da3f4-6adf-4f35-87be-6bf37fc8fc4a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:20:52.255128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125265d-30fc-4562-91c2-94b5aefc0e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:20:52.255739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125265d-30fc-4562-91c2-94b5aefc0e13 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:52.256014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125265d-30fc-4562-91c2-94b5aefc0e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:52.256284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125265d-30fc-4562-91c2-94b5aefc0e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:52.257905 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4125265d-30fc-4562-91c2-94b5aefc0e13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:52.377565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125265d-30fc-4562-91c2-94b5aefc0e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:20:52.378965 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125265d-30fc-4562-91c2-94b5aefc0e13 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:20:52.400167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 667/1340] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:20:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:20:56.488500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132dbfe-a45e-49c9-8a07-49f2ba00ab83 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:56.489120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132dbfe-a45e-49c9-8a07-49f2ba00ab83 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:56.489120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132dbfe-a45e-49c9-8a07-49f2ba00ab83 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:56.490584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 674/1341] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:20:56.495923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fe079-0777-436e-a5ef-55d0e7dcecd4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:56.496373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fe079-0777-436e-a5ef-55d0e7dcecd4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:56.496472 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fe079-0777-436e-a5ef-55d0e7dcecd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:56.502220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:56.565641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-8d1fe079-0777-436e-a5ef-55d0e7dcecd4 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:20:56.566770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8d1fe079-0777-436e-a5ef-55d0e7dcecd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:56.669646 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 668/1342] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:20:56.948867 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132dbfe-a45e-49c9-8a07-49f2ba00ab83 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:20:56.950432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e132dbfe-a45e-49c9-8a07-49f2ba00ab83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:57.003912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:57.063237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132dbfe-a45e-49c9-8a07-49f2ba00ab83 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:20:57.136929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc4450-967b-4b79-93aa-faf11ab20414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:20:57.137552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc4450-967b-4b79-93aa-faf11ab20414 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:20:57.137822 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc4450-967b-4b79-93aa-faf11ab20414 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:20:57.138094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc4450-967b-4b79-93aa-faf11ab20414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:20:57.139671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-44bc4450-967b-4b79-93aa-faf11ab20414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:20:57.233746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:20:57.250857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc4450-967b-4b79-93aa-faf11ab20414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:20:57.252286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc4450-967b-4b79-93aa-faf11ab20414 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:20:57.271267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 675/1343] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:00.150647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a08843-49a9-40d1-94ae-079c757e91d0 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:00.151242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a08843-49a9-40d1-94ae-079c757e91d0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:00.151242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a08843-49a9-40d1-94ae-079c757e91d0 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:00.152571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 669/1344] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:00.158011 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da8249-85d0-46fb-9eec-76cccdaabfcc None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:00.158426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da8249-85d0-46fb-9eec-76cccdaabfcc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:00.158577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da8249-85d0-46fb-9eec-76cccdaabfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:00.230176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-19da8249-85d0-46fb-9eec-76cccdaabfcc None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:00.231375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-19da8249-85d0-46fb-9eec-76cccdaabfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:00.335603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 676/1345] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:00.613974 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a08843-49a9-40d1-94ae-079c757e91d0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:00.615538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-25a08843-49a9-40d1-94ae-079c757e91d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:00.710721 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:00.727989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a08843-49a9-40d1-94ae-079c757e91d0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:00.802577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786c8ce-ae08-4172-8741-5482b8268cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:00.803179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786c8ce-ae08-4172-8741-5482b8268cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:00.803441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786c8ce-ae08-4172-8741-5482b8268cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:00.803699 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786c8ce-ae08-4172-8741-5482b8268cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:00.805340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a786c8ce-ae08-4172-8741-5482b8268cf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:00.917361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786c8ce-ae08-4172-8741-5482b8268cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:00.918843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786c8ce-ae08-4172-8741-5482b8268cf7 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:00.937381 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 670/1346] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:05.087376 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d4992-e445-44ab-9ee9-d9f0cb966c65 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:05.087911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d4992-e445-44ab-9ee9-d9f0cb966c65 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:05.087911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d4992-e445-44ab-9ee9-d9f0cb966c65 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:05.089333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 677/1347] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:05.094967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e7a4e-b33d-4748-a9b1-37984cf2bacb None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:05.095424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e7a4e-b33d-4748-a9b1-37984cf2bacb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:05.095531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e7a4e-b33d-4748-a9b1-37984cf2bacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:05.112014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:05.165596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-652e7a4e-b33d-4748-a9b1-37984cf2bacb None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:05.166643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-652e7a4e-b33d-4748-a9b1-37984cf2bacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:05.273141 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 671/1348] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:05.552897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d4992-e445-44ab-9ee9-d9f0cb966c65 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:05.555465 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6a9d4992-e445-44ab-9ee9-d9f0cb966c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:05.620979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:05.672595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d4992-e445-44ab-9ee9-d9f0cb966c65 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:05.748806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26285d-9135-4d52-a47d-5f0c34db7bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:05.749438 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26285d-9135-4d52-a47d-5f0c34db7bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:05.749708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26285d-9135-4d52-a47d-5f0c34db7bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:05.749980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26285d-9135-4d52-a47d-5f0c34db7bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:05.751531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eb26285d-9135-4d52-a47d-5f0c34db7bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:05.866746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:05.870754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26285d-9135-4d52-a47d-5f0c34db7bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:05.872496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26285d-9135-4d52-a47d-5f0c34db7bc6 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:05.890599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 678/1349] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:10.812129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382045e9-7a83-46bc-bccc-107b99795f17 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:10.812730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382045e9-7a83-46bc-bccc-107b99795f17 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:10.812730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382045e9-7a83-46bc-bccc-107b99795f17 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:10.814279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 672/1350] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:10.824406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583a693-dc47-41f2-98d5-f44780cd0d84 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:10.825676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583a693-dc47-41f2-98d5-f44780cd0d84 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:10.826278 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583a693-dc47-41f2-98d5-f44780cd0d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:10.961418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d583a693-dc47-41f2-98d5-f44780cd0d84 None None] MFA Rules not processed for user `e6d3b7887c13466ab8e3d9737f907c56`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:10.963543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d583a693-dc47-41f2-98d5-f44780cd0d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:11.106170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 679/1351] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:21:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:15.061514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b0836-f309-4ff1-97a9-9dcd8ffe0c9f None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:15.062087 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b0836-f309-4ff1-97a9-9dcd8ffe0c9f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:15.062087 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b0836-f309-4ff1-97a9-9dcd8ffe0c9f None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:15.063669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 673/1352] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:15.068495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928de443-10d9-4f62-b7a0-30d3fc4b9e07 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:15.069359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928de443-10d9-4f62-b7a0-30d3fc4b9e07 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:15.069840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928de443-10d9-4f62-b7a0-30d3fc4b9e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:15.126694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:15.153459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-928de443-10d9-4f62-b7a0-30d3fc4b9e07 None None] MFA Rules not processed for user `e6d3b7887c13466ab8e3d9737f907c56`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:15.155276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-928de443-10d9-4f62-b7a0-30d3fc4b9e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:15.271549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=59 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:15.277836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 680/1353] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:21:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:16.808626 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0c246-8ed5-4f94-bfad-6dd8caab9f3f None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:16.809185 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0c246-8ed5-4f94-bfad-6dd8caab9f3f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:16.809454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0c246-8ed5-4f94-bfad-6dd8caab9f3f None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:16.810867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 674/1354] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:16.816278 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab890f97-4cb3-411e-83ac-fcb94063582f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:16.816729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab890f97-4cb3-411e-83ac-fcb94063582f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:16.816821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab890f97-4cb3-411e-83ac-fcb94063582f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:16.894508 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ab890f97-4cb3-411e-83ac-fcb94063582f None None] MFA Rules not processed for user `e6d3b7887c13466ab8e3d9737f907c56`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:16.895694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ab890f97-4cb3-411e-83ac-fcb94063582f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:16.938940 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:17.007175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 681/1355] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:21:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:21:20.561076 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52312d0-7c37-49be-83d8-e6a19395af37 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:20.561715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52312d0-7c37-49be-83d8-e6a19395af37 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:20.561715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52312d0-7c37-49be-83d8-e6a19395af37 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:20.562965 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 675/1356] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:20.569308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb241f-09b0-48bb-adbb-be3e7d0dfdcf None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:20.570301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb241f-09b0-48bb-adbb-be3e7d0dfdcf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:20.570821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb241f-09b0-48bb-adbb-be3e7d0dfdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:20.643762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b6bb241f-09b0-48bb-adbb-be3e7d0dfdcf None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:20.644855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b6bb241f-09b0-48bb-adbb-be3e7d0dfdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:20.752024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 682/1357] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:21.049271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a7e7e-6e83-49ec-a0a3-921edd062243 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:21.049494 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a7e7e-6e83-49ec-a0a3-921edd062243 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:21.049943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a7e7e-6e83-49ec-a0a3-921edd062243 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:21.051418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 676/1358] 10.222.0.60 () {54 vars in 965 bytes} [Sat Sep 17 04:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:21.061673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caac610-04c2-4696-b7e1-9dbf9036884d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:21.062090 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caac610-04c2-4696-b7e1-9dbf9036884d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:21.062503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caac610-04c2-4696-b7e1-9dbf9036884d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:21.080443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:21.145062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3caac610-04c2-4696-b7e1-9dbf9036884d None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:21.146422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3caac610-04c2-4696-b7e1-9dbf9036884d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:21.276414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 683/1359] 10.222.0.60 () {58 vars in 1062 bytes} [Sat Sep 17 04:21:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:21.284577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a7e7e-6e83-49ec-a0a3-921edd062243 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:21.286192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-da1a7e7e-6e83-49ec-a0a3-921edd062243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:21.442326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a7e7e-6e83-49ec-a0a3-921edd062243 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:21.519347 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:21.561435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3656b-3aa9-46a0-984c-c3814f40ab16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:21.562125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3656b-3aa9-46a0-984c-c3814f40ab16 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:21.562499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3656b-3aa9-46a0-984c-c3814f40ab16 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:21.562868 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3656b-3aa9-46a0-984c-c3814f40ab16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:21.564645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5bf3656b-3aa9-46a0-984c-c3814f40ab16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:21.668081 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3656b-3aa9-46a0-984c-c3814f40ab16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:21.670330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3656b-3aa9-46a0-984c-c3814f40ab16 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:21.691993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 677/1360] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:21:24.042934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20733b62-01c3-4736-ae29-7f4c8cd2c53e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:24.042934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20733b62-01c3-4736-ae29-7f4c8cd2c53e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:24.042934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20733b62-01c3-4736-ae29-7f4c8cd2c53e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:24.044577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 684/1361] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:24.049591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5ad53-aa3e-491d-824b-bb2f4b428580 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:24.050061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5ad53-aa3e-491d-824b-bb2f4b428580 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:24.050061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5ad53-aa3e-491d-824b-bb2f4b428580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:24.116924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1af5ad53-aa3e-491d-824b-bb2f4b428580 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:24.118645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1af5ad53-aa3e-491d-824b-bb2f4b428580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:24.157111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:24.225492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 678/1362] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:24.517109 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20733b62-01c3-4736-ae29-7f4c8cd2c53e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:24.519338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-20733b62-01c3-4736-ae29-7f4c8cd2c53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:24.602223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:24.716497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20733b62-01c3-4736-ae29-7f4c8cd2c53e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:24.857646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91221490-98a4-4229-a4dc-bb33f74bc7fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:24.859015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91221490-98a4-4229-a4dc-bb33f74bc7fe service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:24.859292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91221490-98a4-4229-a4dc-bb33f74bc7fe service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:24.860056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91221490-98a4-4229-a4dc-bb33f74bc7fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:24.861684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-91221490-98a4-4229-a4dc-bb33f74bc7fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:24.885351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:24.952588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91221490-98a4-4229-a4dc-bb33f74bc7fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:24.954220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91221490-98a4-4229-a4dc-bb33f74bc7fe service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:24.972222 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 685/1363] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:27.461237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5b5a3-271c-4a84-86f3-ac623c81b5f7 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:27.461812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5b5a3-271c-4a84-86f3-ac623c81b5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:27.461812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5b5a3-271c-4a84-86f3-ac623c81b5f7 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:27.463351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 679/1364] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:27.468845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7107f-3eb7-4376-b804-cb87c9690286 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:27.469305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7107f-3eb7-4376-b804-cb87c9690286 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:27.469605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7107f-3eb7-4376-b804-cb87c9690286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:27.538468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-96c7107f-3eb7-4376-b804-cb87c9690286 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:27.539912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-96c7107f-3eb7-4376-b804-cb87c9690286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:27.641449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:27.650716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 686/1365] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:27.928513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf27a0-3be7-4eb5-901c-e7a3e6caa64c None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:27.928846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf27a0-3be7-4eb5-901c-e7a3e6caa64c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:27.929153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf27a0-3be7-4eb5-901c-e7a3e6caa64c None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:27.931050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 680/1366] 10.222.0.60 () {54 vars in 965 bytes} [Sat Sep 17 04:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:27.936313 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f5aa9-28d1-4b1b-be88-62a0e62811ce None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:27.936574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f5aa9-28d1-4b1b-be88-62a0e62811ce None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:27.936864 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f5aa9-28d1-4b1b-be88-62a0e62811ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:28.020264 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f22f5aa9-28d1-4b1b-be88-62a0e62811ce None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:28.021492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f22f5aa9-28d1-4b1b-be88-62a0e62811ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:28.140443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 687/1367] 10.222.0.60 () {58 vars in 1062 bytes} [Sat Sep 17 04:21:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:21:28.149020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf27a0-3be7-4eb5-901c-e7a3e6caa64c None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:28.150541 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-26bf27a0-3be7-4eb5-901c-e7a3e6caa64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:28.300661 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf27a0-3be7-4eb5-901c-e7a3e6caa64c None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:28.321728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:28.415425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc55bfa-7227-456b-b055-ed996b8b9d50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:28.416020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc55bfa-7227-456b-b055-ed996b8b9d50 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:28.416268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc55bfa-7227-456b-b055-ed996b8b9d50 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:28.416539 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc55bfa-7227-456b-b055-ed996b8b9d50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:28.418163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-adc55bfa-7227-456b-b055-ed996b8b9d50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:28.510563 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc55bfa-7227-456b-b055-ed996b8b9d50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:28.512210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc55bfa-7227-456b-b055-ed996b8b9d50 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:28.533447 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 681/1368] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:21:28.623643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f6e4d-36a6-4dfe-bd1a-18f7980d394d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:28.624215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f6e4d-36a6-4dfe-bd1a-18f7980d394d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:28.624215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f6e4d-36a6-4dfe-bd1a-18f7980d394d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:28.625745 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 688/1369] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:21:28.631569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90be761-0cbf-4528-938a-c866f7822617 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:28.632053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90be761-0cbf-4528-938a-c866f7822617 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:28.633106 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90be761-0cbf-4528-938a-c866f7822617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:28.678990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:28.721980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f90be761-0cbf-4528-938a-c866f7822617 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:28.723229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f90be761-0cbf-4528-938a-c866f7822617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:28.846777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 682/1370] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:29.138219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00974298-f4cd-46b4-95c5-db5c1d1509f1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:29.138504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00974298-f4cd-46b4-95c5-db5c1d1509f1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:29.138772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00974298-f4cd-46b4-95c5-db5c1d1509f1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:29.140401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 689/1371] 10.222.0.60 () {54 vars in 964 bytes} [Sat Sep 17 04:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:29.145544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2693c5-0fa6-46d0-a44f-7d73f3995e64 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:29.145841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2693c5-0fa6-46d0-a44f-7d73f3995e64 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:29.146105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2693c5-0fa6-46d0-a44f-7d73f3995e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:29.224455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ec2693c5-0fa6-46d0-a44f-7d73f3995e64 None None] MFA Rules not processed for user `6adf758d5fb9430189ee9057b4b7ea97`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:29.225690 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ec2693c5-0fa6-46d0-a44f-7d73f3995e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:29.278677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:29.318896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 683/1372] 10.222.0.60 () {58 vars in 1061 bytes} [Sat Sep 17 04:21:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:29.326817 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00974298-f4cd-46b4-95c5-db5c1d1509f1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:29.328322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-00974298-f4cd-46b4-95c5-db5c1d1509f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:29.382850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:29.439231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00974298-f4cd-46b4-95c5-db5c1d1509f1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:29.520905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ba3d2-0510-420a-b190-f619efd13d56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:29.520905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ba3d2-0510-420a-b190-f619efd13d56 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:29.520905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ba3d2-0510-420a-b190-f619efd13d56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:29.520905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ba3d2-0510-420a-b190-f619efd13d56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:29.521654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8b8ba3d2-0510-420a-b190-f619efd13d56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:29.657401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:29.682618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ba3d2-0510-420a-b190-f619efd13d56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:29.684682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ba3d2-0510-420a-b190-f619efd13d56 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:29.710700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 690/1373] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:30.803947 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e06b8c-1e78-4f43-b9cf-4ceda74a934a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:30.804543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e06b8c-1e78-4f43-b9cf-4ceda74a934a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:30.804543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e06b8c-1e78-4f43-b9cf-4ceda74a934a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:30.805928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 684/1374] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:21:30.811443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09ee8b-37a7-4753-ab3c-0f9a8d40a7a6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:30.811889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09ee8b-37a7-4753-ab3c-0f9a8d40a7a6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:30.811987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09ee8b-37a7-4753-ab3c-0f9a8d40a7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:30.880671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ce09ee8b-37a7-4753-ab3c-0f9a8d40a7a6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:30.882098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ce09ee8b-37a7-4753-ab3c-0f9a8d40a7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:30.980986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 691/1375] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:31.270880 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e06b8c-1e78-4f43-b9cf-4ceda74a934a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:31.272442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-96e06b8c-1e78-4f43-b9cf-4ceda74a934a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:31.444901 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e06b8c-1e78-4f43-b9cf-4ceda74a934a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:31.499834 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:31.563459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddadfc-5aac-41be-be40-90478f082bd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:31.564072 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddadfc-5aac-41be-be40-90478f082bd6 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:31.564346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddadfc-5aac-41be-be40-90478f082bd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:31.564612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddadfc-5aac-41be-be40-90478f082bd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:31.566253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b5ddadfc-5aac-41be-be40-90478f082bd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:31.659580 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddadfc-5aac-41be-be40-90478f082bd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:31.661274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddadfc-5aac-41be-be40-90478f082bd6 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:31.680057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 685/1376] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:33.873999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de884a84-68c5-4498-b616-905b9449921b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:33.874537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de884a84-68c5-4498-b616-905b9449921b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:33.874537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de884a84-68c5-4498-b616-905b9449921b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:33.875899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 692/1377] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:33.881236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1cd27-8952-4221-8e1b-312d448461be None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:33.881697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1cd27-8952-4221-8e1b-312d448461be None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:33.881783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1cd27-8952-4221-8e1b-312d448461be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:33.948619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1be1cd27-8952-4221-8e1b-312d448461be None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:33.949826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1be1cd27-8952-4221-8e1b-312d448461be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:33.957277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:34.057550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 686/1378] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:34.342286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de884a84-68c5-4498-b616-905b9449921b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:34.343836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-de884a84-68c5-4498-b616-905b9449921b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:34.408928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:34.507153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de884a84-68c5-4498-b616-905b9449921b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:34.630877 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67873b5d-f5b9-4ea8-a62c-42c95a2872e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:34.631394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67873b5d-f5b9-4ea8-a62c-42c95a2872e2 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:34.631638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67873b5d-f5b9-4ea8-a62c-42c95a2872e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:34.631910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67873b5d-f5b9-4ea8-a62c-42c95a2872e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:34.633605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-67873b5d-f5b9-4ea8-a62c-42c95a2872e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:34.654937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:34.741277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67873b5d-f5b9-4ea8-a62c-42c95a2872e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:34.742957 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67873b5d-f5b9-4ea8-a62c-42c95a2872e2 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:34.762641 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 693/1379] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:21:36.958985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97dded1-485e-4791-9377-4359ce2e7eaf None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:36.959462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97dded1-485e-4791-9377-4359ce2e7eaf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:36.959559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97dded1-485e-4791-9377-4359ce2e7eaf None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:36.961128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 687/1380] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:36.966295 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623d175-1fb3-4dba-a037-3445273ecd63 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:36.966755 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623d175-1fb3-4dba-a037-3445273ecd63 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:36.966958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623d175-1fb3-4dba-a037-3445273ecd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:37.036481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4623d175-1fb3-4dba-a037-3445273ecd63 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:37.037860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4623d175-1fb3-4dba-a037-3445273ecd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:37.134028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:37.145313 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 694/1381] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:37.431099 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97dded1-485e-4791-9377-4359ce2e7eaf None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:37.432686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c97dded1-485e-4791-9377-4359ce2e7eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:37.589134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:37.600850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97dded1-485e-4791-9377-4359ce2e7eaf None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:37.718650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c74f9-8ef5-4b94-b9a9-cafac5b26f66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:37.719174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c74f9-8ef5-4b94-b9a9-cafac5b26f66 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:37.719451 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c74f9-8ef5-4b94-b9a9-cafac5b26f66 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:37.719723 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c74f9-8ef5-4b94-b9a9-cafac5b26f66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:37.721420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-639c74f9-8ef5-4b94-b9a9-cafac5b26f66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:37.821437 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c74f9-8ef5-4b94-b9a9-cafac5b26f66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:37.822939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c74f9-8ef5-4b94-b9a9-cafac5b26f66 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:37.833510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:37.843400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 688/1382] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:40.101150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c507d9f-3466-4584-a1dd-07377b3a613f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:40.101847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c507d9f-3466-4584-a1dd-07377b3a613f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:40.101847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c507d9f-3466-4584-a1dd-07377b3a613f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:40.103249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 695/1383] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:21:40.109923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf57b21-dbba-45f1-a4a2-329c566e76bf None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:40.110410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf57b21-dbba-45f1-a4a2-329c566e76bf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:40.110640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf57b21-dbba-45f1-a4a2-329c566e76bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:40.186914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9bf57b21-dbba-45f1-a4a2-329c566e76bf None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:40.188298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9bf57b21-dbba-45f1-a4a2-329c566e76bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:40.298454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 689/1384] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:40.587937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c507d9f-3466-4584-a1dd-07377b3a613f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:40.589678 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7c507d9f-3466-4584-a1dd-07377b3a613f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:40.754988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c507d9f-3466-4584-a1dd-07377b3a613f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:40.841104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:40.891166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1e6b8-99cf-4352-ac56-9db68b4e65a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:40.891773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1e6b8-99cf-4352-ac56-9db68b4e65a7 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:40.892047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1e6b8-99cf-4352-ac56-9db68b4e65a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:40.892313 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1e6b8-99cf-4352-ac56-9db68b4e65a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:40.894056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-acd1e6b8-99cf-4352-ac56-9db68b4e65a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:40.992588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1e6b8-99cf-4352-ac56-9db68b4e65a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:40.994887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1e6b8-99cf-4352-ac56-9db68b4e65a7 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:41.018630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 696/1385] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:43.231491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0dcf3-bfeb-45a1-a1bd-79a2f0bf4b44 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:43.232082 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0dcf3-bfeb-45a1-a1bd-79a2f0bf4b44 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:43.232279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0dcf3-bfeb-45a1-a1bd-79a2f0bf4b44 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:43.233658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 690/1386] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:43.239444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889d551-a73c-46d1-9dc1-7fdd58a2bff5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:43.239921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889d551-a73c-46d1-9dc1-7fdd58a2bff5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:43.240233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889d551-a73c-46d1-9dc1-7fdd58a2bff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:43.321564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5889d551-a73c-46d1-9dc1-7fdd58a2bff5 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:43.322790 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5889d551-a73c-46d1-9dc1-7fdd58a2bff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:43.356289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:43.419483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 697/1387] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:43.703286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0dcf3-bfeb-45a1-a1bd-79a2f0bf4b44 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:43.704812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e0e0dcf3-bfeb-45a1-a1bd-79a2f0bf4b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:43.788232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:43.860817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0dcf3-bfeb-45a1-a1bd-79a2f0bf4b44 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:43.975195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953f261-df69-4b4f-b927-fac3087ae088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:43.975721 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953f261-df69-4b4f-b927-fac3087ae088 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:43.975996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953f261-df69-4b4f-b927-fac3087ae088 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:43.976335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953f261-df69-4b4f-b927-fac3087ae088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:43.977967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5953f261-df69-4b4f-b927-fac3087ae088 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:44.017920 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:44.069567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953f261-df69-4b4f-b927-fac3087ae088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:44.071451 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953f261-df69-4b4f-b927-fac3087ae088 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:44.089808 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 691/1388] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:44.877710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48886f28-20ca-44e3-91f6-1ca7db43ce2b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:44.878261 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48886f28-20ca-44e3-91f6-1ca7db43ce2b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:44.878261 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48886f28-20ca-44e3-91f6-1ca7db43ce2b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:44.879617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 698/1389] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:44.885122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dbb36-2506-4f13-b5d9-d5a1eac23c8a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:44.885921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dbb36-2506-4f13-b5d9-d5a1eac23c8a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:44.886204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dbb36-2506-4f13-b5d9-d5a1eac23c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:44.952638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-284dbb36-2506-4f13-b5d9-d5a1eac23c8a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:44.953782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-284dbb36-2506-4f13-b5d9-d5a1eac23c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:45.064106 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 692/1390] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:45.364907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2c4f8-1737-48fd-a688-8ff62ad9ebcb None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:45.365222 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2c4f8-1737-48fd-a688-8ff62ad9ebcb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:45.365482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2c4f8-1737-48fd-a688-8ff62ad9ebcb None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:45.367035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 699/1391] 10.222.0.60 () {54 vars in 962 bytes} [Sat Sep 17 04:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:45.372214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d85cfe-bdda-4f30-addb-f97c28e09600 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:45.372503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d85cfe-bdda-4f30-addb-f97c28e09600 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:45.372779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d85cfe-bdda-4f30-addb-f97c28e09600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:45.389280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:45.452367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d0d85cfe-bdda-4f30-addb-f97c28e09600 None None] MFA Rules not processed for user `acf8a0ae03c94aa78cc2ab0e4e1c87b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:45.453605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d0d85cfe-bdda-4f30-addb-f97c28e09600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:45.530765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 693/1392] 10.222.0.60 () {58 vars in 1059 bytes} [Sat Sep 17 04:21:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:45.540180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2c4f8-1737-48fd-a688-8ff62ad9ebcb None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:45.541768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7eb2c4f8-1737-48fd-a688-8ff62ad9ebcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:45.662856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2c4f8-1737-48fd-a688-8ff62ad9ebcb None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:45.720861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:45.736652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ac254-424e-4aa6-8db1-5d917d15d992 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf8a0ae03c94aa78cc2ab0e4e1c87b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:45.737317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ac254-424e-4aa6-8db1-5d917d15d992 service glance] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:45.737591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ac254-424e-4aa6-8db1-5d917d15d992 service glance] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:45.737859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ac254-424e-4aa6-8db1-5d917d15d992 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:45.739452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f19ac254-424e-4aa6-8db1-5d917d15d992 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:45.848359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ac254-424e-4aa6-8db1-5d917d15d992 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:45.850162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ac254-424e-4aa6-8db1-5d917d15d992 service glance] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:45.868804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 700/1393] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:46.305899 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a660e-89b9-454c-91a7-5c0e1eb3b39a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:46.306517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a660e-89b9-454c-91a7-5c0e1eb3b39a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:46.306517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a660e-89b9-454c-91a7-5c0e1eb3b39a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:46.307780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 694/1394] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:46.313361 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759335c2-0fda-47e2-a910-0b57f6fb6556 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:46.313908 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759335c2-0fda-47e2-a910-0b57f6fb6556 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:46.313908 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759335c2-0fda-47e2-a910-0b57f6fb6556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:46.386231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-759335c2-0fda-47e2-a910-0b57f6fb6556 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:46.387448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-759335c2-0fda-47e2-a910-0b57f6fb6556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:46.433112 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:46.506606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 701/1395] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:21:46.638740 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a823b-c970-4842-a7ec-a7629a3c6c9d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:46.639014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a823b-c970-4842-a7ec-a7629a3c6c9d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:46.639285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a823b-c970-4842-a7ec-a7629a3c6c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:46.711021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-971a823b-c970-4842-a7ec-a7629a3c6c9d None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:46.712246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-971a823b-c970-4842-a7ec-a7629a3c6c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:46.771417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:46.805505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 695/1396] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:21:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:46.813571 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759335c2-0fda-47e2-a910-0b57f6fb6556 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:46.815136 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-759335c2-0fda-47e2-a910-0b57f6fb6556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:46.848325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bcef6-fe1b-4ee0-aafd-1d334fcddac7 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:46.848674 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bcef6-fe1b-4ee0-aafd-1d334fcddac7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:46.848954 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bcef6-fe1b-4ee0-aafd-1d334fcddac7 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:46.850358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 696/1397] 10.222.0.60 () {54 vars in 943 bytes} [Sat Sep 17 04:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:21:46.856911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c5ce4-6826-4555-8eaa-6c681df5a70e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:46.857195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c5ce4-6826-4555-8eaa-6c681df5a70e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:46.857471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c5ce4-6826-4555-8eaa-6c681df5a70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:46.934911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759335c2-0fda-47e2-a910-0b57f6fb6556 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:46.936483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0e3c5ce4-6826-4555-8eaa-6c681df5a70e None None] MFA Rules not processed for user `7aac7cd1742449328facf2902483f259`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:46.937764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0e3c5ce4-6826-4555-8eaa-6c681df5a70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:46.989531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:47.047137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2aa35-ef07-4717-b1e3-68b8c681af55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:47.047418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 697/1398] 10.222.0.60 () {58 vars in 1040 bytes} [Sat Sep 17 04:21:46 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:47.047831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2aa35-ef07-4717-b1e3-68b8c681af55 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:47.048109 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2aa35-ef07-4717-b1e3-68b8c681af55 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:47.048364 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2aa35-ef07-4717-b1e3-68b8c681af55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:47.050076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-85d2aa35-ef07-4717-b1e3-68b8c681af55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:47.056833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c5ce4-6826-4555-8eaa-6c681df5a70e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:47.067081 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c5ce4-6826-4555-8eaa-6c681df5a70e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:47.096162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:47.154639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2aa35-ef07-4717-b1e3-68b8c681af55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:47.157224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2aa35-ef07-4717-b1e3-68b8c681af55 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:47.177791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf72b32-91e1-4c45-8989-e260cbbb1ea1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:47.178212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf72b32-91e1-4c45-8989-e260cbbb1ea1 service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:47.178489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf72b32-91e1-4c45-8989-e260cbbb1ea1 service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:47.178764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf72b32-91e1-4c45-8989-e260cbbb1ea1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:47.180482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-daf72b32-91e1-4c45-8989-e260cbbb1ea1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:47.181901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 702/1399] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:47.274221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf72b32-91e1-4c45-8989-e260cbbb1ea1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:47.275854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf72b32-91e1-4c45-8989-e260cbbb1ea1 service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:47.285127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 698/1400] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:21:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 04:21:49.382866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95477b1-0095-4ec3-88ee-71ad0c755d37 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:49.383540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95477b1-0095-4ec3-88ee-71ad0c755d37 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:49.383540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95477b1-0095-4ec3-88ee-71ad0c755d37 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:49.384911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 703/1401] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:49.390454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf3f96-33b6-4245-a7ba-13ebc8675ea3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:49.390977 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf3f96-33b6-4245-a7ba-13ebc8675ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:49.390977 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf3f96-33b6-4245-a7ba-13ebc8675ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:49.465387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-bcaf3f96-33b6-4245-a7ba-13ebc8675ea3 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:49.466675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bcaf3f96-33b6-4245-a7ba-13ebc8675ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:49.473522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:49.577905 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 699/1402] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:49.870327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95477b1-0095-4ec3-88ee-71ad0c755d37 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:49.872309 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b95477b1-0095-4ec3-88ee-71ad0c755d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:49.940665 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:50.029049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95477b1-0095-4ec3-88ee-71ad0c755d37 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:50.150965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051f799-7a79-4de8-81ae-18f08129c27d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:50.151598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051f799-7a79-4de8-81ae-18f08129c27d service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:50.151906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051f799-7a79-4de8-81ae-18f08129c27d service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:50.152177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051f799-7a79-4de8-81ae-18f08129c27d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:50.153814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0051f799-7a79-4de8-81ae-18f08129c27d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:50.186065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:50.253689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051f799-7a79-4de8-81ae-18f08129c27d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:50.255594 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051f799-7a79-4de8-81ae-18f08129c27d service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:50.276854 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 704/1403] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:52.493230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fbd1b-f2b2-42e3-a880-3a39518e1f27 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:52.493824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fbd1b-f2b2-42e3-a880-3a39518e1f27 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:52.493824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fbd1b-f2b2-42e3-a880-3a39518e1f27 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:52.495112 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 700/1404] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:52.500525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dac7b-20ed-4fce-8310-0b0b4ec8bbd1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:52.500973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dac7b-20ed-4fce-8310-0b0b4ec8bbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:52.501174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dac7b-20ed-4fce-8310-0b0b4ec8bbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:52.576856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-db9dac7b-20ed-4fce-8310-0b0b4ec8bbd1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:52.578101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-db9dac7b-20ed-4fce-8310-0b0b4ec8bbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:52.694861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 705/1405] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:52.983251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fbd1b-f2b2-42e3-a880-3a39518e1f27 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:52.985163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-557fbd1b-f2b2-42e3-a880-3a39518e1f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:53.131042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:53.142364 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fbd1b-f2b2-42e3-a880-3a39518e1f27 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:53.265548 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2703f8-b1d6-4af2-a264-c318765b7774 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:53.266070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2703f8-b1d6-4af2-a264-c318765b7774 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:53.266344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2703f8-b1d6-4af2-a264-c318765b7774 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:53.266610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2703f8-b1d6-4af2-a264-c318765b7774 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:53.268237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ff2703f8-b1d6-4af2-a264-c318765b7774 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:53.366010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2703f8-b1d6-4af2-a264-c318765b7774 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:53.367769 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2703f8-b1d6-4af2-a264-c318765b7774 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:53.378967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:53.388532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 701/1406] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:21:55.596119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d7026-a101-4784-876e-aee28eff27bc None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:55.597045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d7026-a101-4784-876e-aee28eff27bc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:55.597603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d7026-a101-4784-876e-aee28eff27bc None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:55.599124 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 706/1407] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:21:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:21:55.604856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c186e10-b546-420b-9fca-cb071525b75e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:55.605660 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c186e10-b546-420b-9fca-cb071525b75e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:55.605936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c186e10-b546-420b-9fca-cb071525b75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:55.685820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2c186e10-b546-420b-9fca-cb071525b75e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:21:55.687670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2c186e10-b546-420b-9fca-cb071525b75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:55.792345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 702/1408] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:21:56.088011 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d7026-a101-4784-876e-aee28eff27bc None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:21:56.089673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-931d7026-a101-4784-876e-aee28eff27bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:56.092827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:56.256748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d7026-a101-4784-876e-aee28eff27bc None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:21:56.338511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:56.383034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bb8dd-5ca1-4b73-902a-f8afd49a3aba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:21:56.383646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb8dd-5ca1-4b73-902a-f8afd49a3aba service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:21:56.383919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb8dd-5ca1-4b73-902a-f8afd49a3aba service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:21:56.384186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb8dd-5ca1-4b73-902a-f8afd49a3aba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:21:56.385819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5d5bb8dd-5ca1-4b73-902a-f8afd49a3aba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:21:56.481580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bb8dd-5ca1-4b73-902a-f8afd49a3aba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:21:56.483184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bb8dd-5ca1-4b73-902a-f8afd49a3aba service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:21:56.501869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 707/1409] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:05.790854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=67 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:06.499256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=61 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:10.010540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f1a1-5702-4b09-8795-f23abb11af4a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:10.011088 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f1a1-5702-4b09-8795-f23abb11af4a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:10.011088 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f1a1-5702-4b09-8795-f23abb11af4a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:10.012452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 703/1410] 10.222.0.60 () {54 vars in 918 bytes} [Sat Sep 17 04:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:10.017838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280ab5c-d2f9-4dbc-a340-7dd28be40a34 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:10.018285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280ab5c-d2f9-4dbc-a340-7dd28be40a34 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:10.018404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280ab5c-d2f9-4dbc-a340-7dd28be40a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:10.096499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9280ab5c-d2f9-4dbc-a340-7dd28be40a34 None None] MFA Rules not processed for user `e6d3b7887c13466ab8e3d9737f907c56`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:10.097741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9280ab5c-d2f9-4dbc-a340-7dd28be40a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:10.208633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 708/1411] 10.222.0.60 () {58 vars in 1015 bytes} [Sat Sep 17 04:22:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:10.223573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fff2bd-e161-4b2a-9785-ad45b05ede60 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:10.223855 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fff2bd-e161-4b2a-9785-ad45b05ede60 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:10.224126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fff2bd-e161-4b2a-9785-ad45b05ede60 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:10.225537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 704/1412] 10.222.0.60 () {54 vars in 918 bytes} [Sat Sep 17 04:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:10.230435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b67ba-c3b2-4f55-8a4d-0fcdbae75da1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:10.230722 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b67ba-c3b2-4f55-8a4d-0fcdbae75da1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:10.230995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b67ba-c3b2-4f55-8a4d-0fcdbae75da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:10.306684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:10.310420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-af2b67ba-c3b2-4f55-8a4d-0fcdbae75da1 None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:10.312335 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-af2b67ba-c3b2-4f55-8a4d-0fcdbae75da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:10.434175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 709/1413] 10.222.0.60 () {58 vars in 1015 bytes} [Sat Sep 17 04:22:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:13.021503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48f0b-e6c2-4393-bdb9-c11bc788ee62 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:13.022396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48f0b-e6c2-4393-bdb9-c11bc788ee62 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:13.022396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48f0b-e6c2-4393-bdb9-c11bc788ee62 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:13.024507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:22:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:13.030210 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5516b0f-80f6-433c-b17b-405ca08033cb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:13.030677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5516b0f-80f6-433c-b17b-405ca08033cb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:13.030786 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5516b0f-80f6-433c-b17b-405ca08033cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:13.100692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b5516b0f-80f6-433c-b17b-405ca08033cb None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:13.101917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b5516b0f-80f6-433c-b17b-405ca08033cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:13.160331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:13.208553 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:13.508141 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e48f0b-e6c2-4393-bdb9-c11bc788ee62 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:13.509717 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-33e48f0b-e6c2-4393-bdb9-c11bc788ee62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:13.669891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e48f0b-e6c2-4393-bdb9-c11bc788ee62 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:13.749153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:13.794326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd423e0-921c-4afb-8ad3-3e85b85a68ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:13.794931 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd423e0-921c-4afb-8ad3-3e85b85a68ea service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:13.795201 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd423e0-921c-4afb-8ad3-3e85b85a68ea service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:13.795464 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd423e0-921c-4afb-8ad3-3e85b85a68ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:13.797038 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cbd423e0-921c-4afb-8ad3-3e85b85a68ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:13.886850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd423e0-921c-4afb-8ad3-3e85b85a68ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:13.888418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd423e0-921c-4afb-8ad3-3e85b85a68ea service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:13.906427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 706/1416] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:15.041444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5816a9-3a91-405b-b029-c0d68ada6596 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:15.042349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5816a9-3a91-405b-b029-c0d68ada6596 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:15.042349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5816a9-3a91-405b-b029-c0d68ada6596 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:15.044400 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 04:22:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:15.050921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439a1c1-8d85-42c8-b96a-3f10abc952b2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:15.051363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439a1c1-8d85-42c8-b96a-3f10abc952b2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:15.051481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439a1c1-8d85-42c8-b96a-3f10abc952b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:15.130713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2439a1c1-8d85-42c8-b96a-3f10abc952b2 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:15.131906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2439a1c1-8d85-42c8-b96a-3f10abc952b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:15.159460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:15.237167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 04:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:15.400185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5816a9-3a91-405b-b029-c0d68ada6596 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:15.402581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bf5816a9-3a91-405b-b029-c0d68ada6596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:15.566595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5816a9-3a91-405b-b029-c0d68ada6596 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:15.610590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:15.684834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee9a7f-621a-47d0-96f6-90bc27e16806 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:15.685480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee9a7f-621a-47d0-96f6-90bc27e16806 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:15.685754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee9a7f-621a-47d0-96f6-90bc27e16806 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:15.686018 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee9a7f-621a-47d0-96f6-90bc27e16806 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:15.687599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-88ee9a7f-621a-47d0-96f6-90bc27e16806 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:15.780060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee9a7f-621a-47d0-96f6-90bc27e16806 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:15.781730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee9a7f-621a-47d0-96f6-90bc27e16806 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:15.799946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 712/1419] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:16.945222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd623ed7-3372-41c4-84d9-64d6ccf4ada8 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:16.945792 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd623ed7-3372-41c4-84d9-64d6ccf4ada8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:16.945792 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd623ed7-3372-41c4-84d9-64d6ccf4ada8 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:16.947184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 04:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:16.953843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03025dad-2f4b-4596-81ed-2903108aa1fd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:16.954351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03025dad-2f4b-4596-81ed-2903108aa1fd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:16.954351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03025dad-2f4b-4596-81ed-2903108aa1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:17.021982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:17.045985 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-03025dad-2f4b-4596-81ed-2903108aa1fd None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:17.047206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-03025dad-2f4b-4596-81ed-2903108aa1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:17.158833 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 04:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:17.311268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd623ed7-3372-41c4-84d9-64d6ccf4ada8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:17.312775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bd623ed7-3372-41c4-84d9-64d6ccf4ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:17.470063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd623ed7-3372-41c4-84d9-64d6ccf4ada8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:17.482356 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:17.588801 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7c1ce-9ea1-4e9b-bc6e-4698feb76065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:17.589430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c1ce-9ea1-4e9b-bc6e-4698feb76065 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:17.589696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c1ce-9ea1-4e9b-bc6e-4698feb76065 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:17.589941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c1ce-9ea1-4e9b-bc6e-4698feb76065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:17.591508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-98c7c1ce-9ea1-4e9b-bc6e-4698feb76065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:17.687168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c1ce-9ea1-4e9b-bc6e-4698feb76065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:17.688766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c1ce-9ea1-4e9b-bc6e-4698feb76065 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:17.707345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 709/1422] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:18.824462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05684c-e71d-4d4d-a73a-fe0ae0ddbbfc None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:18.825103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05684c-e71d-4d4d-a73a-fe0ae0ddbbfc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:18.825103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05684c-e71d-4d4d-a73a-fe0ae0ddbbfc None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:18.826390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 04:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:18.833052 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cd84e-315d-4199-8fac-5cdacba80b8c None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:18.833552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cd84e-315d-4199-8fac-5cdacba80b8c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:18.833670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cd84e-315d-4199-8fac-5cdacba80b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:18.863496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:18.925442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-846cd84e-315d-4199-8fac-5cdacba80b8c None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:18.926673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-846cd84e-315d-4199-8fac-5cdacba80b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:19.040761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 04:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:19.194124 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05684c-e71d-4d4d-a73a-fe0ae0ddbbfc None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:19.195585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eb05684c-e71d-4d4d-a73a-fe0ae0ddbbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:19.314239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:19.351188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05684c-e71d-4d4d-a73a-fe0ae0ddbbfc None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:19.467661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a2c46-ddcd-4f53-a8e7-f6e906313287 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:19.468191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a2c46-ddcd-4f53-a8e7-f6e906313287 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:19.468465 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a2c46-ddcd-4f53-a8e7-f6e906313287 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:19.468734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a2c46-ddcd-4f53-a8e7-f6e906313287 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:19.470351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b89a2c46-ddcd-4f53-a8e7-f6e906313287 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:19.558128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:19.574000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a2c46-ddcd-4f53-a8e7-f6e906313287 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:19.576249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a2c46-ddcd-4f53-a8e7-f6e906313287 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:19.598910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 715/1425] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:20.727673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46613b46-84f5-4ec2-9466-4df29a152459 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:20.728469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46613b46-84f5-4ec2-9466-4df29a152459 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:20.728469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46613b46-84f5-4ec2-9466-4df29a152459 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:20.730469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 04:22:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:20.737665 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e31abf-c45b-4e1c-963f-09e22509628f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:20.738125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e31abf-c45b-4e1c-963f-09e22509628f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:20.738230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e31abf-c45b-4e1c-963f-09e22509628f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:20.821457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-85e31abf-c45b-4e1c-963f-09e22509628f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:20.822983 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-85e31abf-c45b-4e1c-963f-09e22509628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:20.926122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 04:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:21.078371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46613b46-84f5-4ec2-9466-4df29a152459 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:21.080702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-46613b46-84f5-4ec2-9466-4df29a152459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:21.176837 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:21.242901 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46613b46-84f5-4ec2-9466-4df29a152459 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:21.364820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dbbf5-a330-42b4-a84b-e515f1774d07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:21.365482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dbbf5-a330-42b4-a84b-e515f1774d07 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:21.365756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dbbf5-a330-42b4-a84b-e515f1774d07 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:21.366025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dbbf5-a330-42b4-a84b-e515f1774d07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:21.367661 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ae2dbbf5-a330-42b4-a84b-e515f1774d07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:21.416439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:21.458866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dbbf5-a330-42b4-a84b-e515f1774d07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:21.460409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dbbf5-a330-42b4-a84b-e515f1774d07 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:21.478770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 712/1428] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:22.523071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5034c-333c-474c-b6b9-9656615c25c4 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:22.523675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5034c-333c-474c-b6b9-9656615c25c4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:22.523675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5034c-333c-474c-b6b9-9656615c25c4 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:22.525508 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 04:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:22.531999 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07b8be-78d3-4606-843c-7da61f6c011b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:22.532443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07b8be-78d3-4606-843c-7da61f6c011b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:22.532551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07b8be-78d3-4606-843c-7da61f6c011b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:22.610889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7e07b8be-78d3-4606-843c-7da61f6c011b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:22.612666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7e07b8be-78d3-4606-843c-7da61f6c011b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:22.722798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 04:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:22:22.876528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5034c-333c-474c-b6b9-9656615c25c4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:22.878873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-44b5034c-333c-474c-b6b9-9656615c25c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:22.935443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:23.034967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5034c-333c-474c-b6b9-9656615c25c4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:23.152108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79dcc4-907b-4e81-987a-45fa6a8c0c32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:23.152636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79dcc4-907b-4e81-987a-45fa6a8c0c32 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:23.152917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79dcc4-907b-4e81-987a-45fa6a8c0c32 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:23.153210 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79dcc4-907b-4e81-987a-45fa6a8c0c32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:23.154772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2f79dcc4-907b-4e81-987a-45fa6a8c0c32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:23.161700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:23.250856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79dcc4-907b-4e81-987a-45fa6a8c0c32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:23.252464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79dcc4-907b-4e81-987a-45fa6a8c0c32 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:23.271618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 718/1431] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:24.323282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408a0f0-3857-4973-83dc-cc8c413b5dc2 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:24.323889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408a0f0-3857-4973-83dc-cc8c413b5dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:24.323889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408a0f0-3857-4973-83dc-cc8c413b5dc2 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:24.325227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 04:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:24.332735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2df77-8bb8-4e34-9c9e-44ad9dee9123 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:24.333220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2df77-8bb8-4e34-9c9e-44ad9dee9123 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:24.333325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2df77-8bb8-4e34-9c9e-44ad9dee9123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:24.415841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e3b2df77-8bb8-4e34-9c9e-44ad9dee9123 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:24.417120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e3b2df77-8bb8-4e34-9c9e-44ad9dee9123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:24.492342 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:24.528769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 04:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:24.678720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408a0f0-3857-4973-83dc-cc8c413b5dc2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:24.681991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7408a0f0-3857-4973-83dc-cc8c413b5dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:24.700719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:24.834160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408a0f0-3857-4973-83dc-cc8c413b5dc2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:24.928035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:24.963002 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d6ff6-1cfe-4730-8f50-8e7e88d0723f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:24.963616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d6ff6-1cfe-4730-8f50-8e7e88d0723f service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:24.963897 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d6ff6-1cfe-4730-8f50-8e7e88d0723f service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:24.964199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d6ff6-1cfe-4730-8f50-8e7e88d0723f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:24.965854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9d3d6ff6-1cfe-4730-8f50-8e7e88d0723f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:25.057736 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d6ff6-1cfe-4730-8f50-8e7e88d0723f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:25.059324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d6ff6-1cfe-4730-8f50-8e7e88d0723f service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:25.079422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 715/1434] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:26.126466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c73c8-a492-41d1-8f81-f3335e4f5ab3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:26.127045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c73c8-a492-41d1-8f81-f3335e4f5ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:26.127045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c73c8-a492-41d1-8f81-f3335e4f5ab3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:26.128375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 04:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:26.134803 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b747100-e8fb-4d4b-becb-00846416b1af None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:26.135250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b747100-e8fb-4d4b-becb-00846416b1af None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:26.135345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b747100-e8fb-4d4b-becb-00846416b1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:26.218522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7b747100-e8fb-4d4b-becb-00846416b1af None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:26.219725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7b747100-e8fb-4d4b-becb-00846416b1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:26.264447 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:26.322741 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 04:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:26.474416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c73c8-a492-41d1-8f81-f3335e4f5ab3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:26.475841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e27c73c8-a492-41d1-8f81-f3335e4f5ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:26.631187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c73c8-a492-41d1-8f81-f3335e4f5ab3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:26.688358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:26.757465 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3ef19-c68a-47b4-b400-ca073780a475 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:26.758156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3ef19-c68a-47b4-b400-ca073780a475 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:26.758432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3ef19-c68a-47b4-b400-ca073780a475 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:26.758703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3ef19-c68a-47b4-b400-ca073780a475 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:26.760950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e7e3ef19-c68a-47b4-b400-ca073780a475 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:26.855045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3ef19-c68a-47b4-b400-ca073780a475 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:26.856640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3ef19-c68a-47b4-b400-ca073780a475 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:26.875299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 721/1437] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:27.927189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8948a-f9e9-40e4-84bd-854bdb12dc11 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:27.928015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8948a-f9e9-40e4-84bd-854bdb12dc11 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:27.928015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8948a-f9e9-40e4-84bd-854bdb12dc11 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:27.930090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 04:22:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:27.940996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3659af-3b39-42ff-b1a5-6fb76cd15e2c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:27.941795 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3659af-3b39-42ff-b1a5-6fb76cd15e2c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:27.941927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3659af-3b39-42ff-b1a5-6fb76cd15e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:28.023299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-be3659af-3b39-42ff-b1a5-6fb76cd15e2c None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:28.024508 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-be3659af-3b39-42ff-b1a5-6fb76cd15e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:28.026863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:28.130894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 04:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:28.278380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8948a-f9e9-40e4-84bd-854bdb12dc11 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:28.280737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c8d8948a-f9e9-40e4-84bd-854bdb12dc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:28.448064 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8948a-f9e9-40e4-84bd-854bdb12dc11 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:28.480243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:28.575871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad815a3-aabf-43f0-b3f7-b045f5a539df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:28.576391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad815a3-aabf-43f0-b3f7-b045f5a539df service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:28.576664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad815a3-aabf-43f0-b3f7-b045f5a539df service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:28.576929 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad815a3-aabf-43f0-b3f7-b045f5a539df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:28.578548 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-dad815a3-aabf-43f0-b3f7-b045f5a539df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:28.670226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad815a3-aabf-43f0-b3f7-b045f5a539df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:28.671891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad815a3-aabf-43f0-b3f7-b045f5a539df service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:28.691668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 718/1440] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:30.735038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc35885-8248-4e8e-aa00-0390aa56571e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:30.735644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc35885-8248-4e8e-aa00-0390aa56571e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:30.735644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc35885-8248-4e8e-aa00-0390aa56571e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:30.736910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:30.743433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc911951-50a6-427d-bf69-5401d9604912 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:30.744194 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc911951-50a6-427d-bf69-5401d9604912 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:30.744351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc911951-50a6-427d-bf69-5401d9604912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:30.790823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:30.821547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cc911951-50a6-427d-bf69-5401d9604912 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:30.822809 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cc911951-50a6-427d-bf69-5401d9604912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:30.924809 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:31.202300 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc35885-8248-4e8e-aa00-0390aa56571e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:31.203856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5dc35885-8248-4e8e-aa00-0390aa56571e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:31.329702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc35885-8248-4e8e-aa00-0390aa56571e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:31.367946 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:31.412215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55a3c5-8075-426c-9859-9a7d0a16fee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:31.412827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55a3c5-8075-426c-9859-9a7d0a16fee1 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:31.413116 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55a3c5-8075-426c-9859-9a7d0a16fee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:31.413391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55a3c5-8075-426c-9859-9a7d0a16fee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:31.414937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fb55a3c5-8075-426c-9859-9a7d0a16fee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:31.649342 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55a3c5-8075-426c-9859-9a7d0a16fee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:31.650817 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55a3c5-8075-426c-9859-9a7d0a16fee1 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:31.669943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 724/1443] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:22:33.984048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768aeb7-51b5-45d5-b434-56c78beee2b8 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:33.985163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768aeb7-51b5-45d5-b434-56c78beee2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:33.985163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768aeb7-51b5-45d5-b434-56c78beee2b8 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:33.987227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:22:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:33.993785 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211a857-0c2b-4a52-b52e-9d50defee155 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:33.994538 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211a857-0c2b-4a52-b52e-9d50defee155 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:33.994730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211a857-0c2b-4a52-b52e-9d50defee155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:34.070214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b211a857-0c2b-4a52-b52e-9d50defee155 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:34.071424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b211a857-0c2b-4a52-b52e-9d50defee155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:34.084658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:34.176824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:34.467990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768aeb7-51b5-45d5-b434-56c78beee2b8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:34.470391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3768aeb7-51b5-45d5-b434-56c78beee2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:34.600745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768aeb7-51b5-45d5-b434-56c78beee2b8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:34.614166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:34.677888 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f98c1b-9b9c-44bb-ba2c-07b44802dc25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:34.678504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f98c1b-9b9c-44bb-ba2c-07b44802dc25 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:34.678780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f98c1b-9b9c-44bb-ba2c-07b44802dc25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:34.679050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f98c1b-9b9c-44bb-ba2c-07b44802dc25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:34.680659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-05f98c1b-9b9c-44bb-ba2c-07b44802dc25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:34.806266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f98c1b-9b9c-44bb-ba2c-07b44802dc25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:34.808429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f98c1b-9b9c-44bb-ba2c-07b44802dc25 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:34.832113 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 721/1446] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:37.324575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034af807-040b-4af8-86e6-0920458d028a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:37.325181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034af807-040b-4af8-86e6-0920458d028a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:37.325181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034af807-040b-4af8-86e6-0920458d028a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:37.326498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:37.332980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909e653-e1f1-48af-8048-0b72ef42d010 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:37.333842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909e653-e1f1-48af-8048-0b72ef42d010 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:37.333842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909e653-e1f1-48af-8048-0b72ef42d010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:37.388159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:37.406897 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9909e653-e1f1-48af-8048-0b72ef42d010 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:37.408184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9909e653-e1f1-48af-8048-0b72ef42d010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:37.511837 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:37.791026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034af807-040b-4af8-86e6-0920458d028a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:37.793212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-034af807-040b-4af8-86e6-0920458d028a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:37.897693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034af807-040b-4af8-86e6-0920458d028a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:37.938768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:37.972854 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21fee6-5290-4f92-bbfc-dc1f2dd689f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:37.973482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21fee6-5290-4f92-bbfc-dc1f2dd689f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:37.973751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21fee6-5290-4f92-bbfc-dc1f2dd689f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:37.974015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21fee6-5290-4f92-bbfc-dc1f2dd689f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:37.975598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-be21fee6-5290-4f92-bbfc-dc1f2dd689f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:38.089471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21fee6-5290-4f92-bbfc-dc1f2dd689f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:38.090827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21fee6-5290-4f92-bbfc-dc1f2dd689f2 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:38.116944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 727/1449] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:40.394476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a18531-4d32-48bf-abc9-1bd59d573e39 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:40.395096 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a18531-4d32-48bf-abc9-1bd59d573e39 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:40.395096 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a18531-4d32-48bf-abc9-1bd59d573e39 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:40.396366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:40.402163 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8741b-3970-4f87-8028-a1ae2ad79487 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:40.402621 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8741b-3970-4f87-8028-a1ae2ad79487 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:40.402731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8741b-3970-4f87-8028-a1ae2ad79487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:40.476389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d0f8741b-3970-4f87-8028-a1ae2ad79487 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:40.477646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d0f8741b-3970-4f87-8028-a1ae2ad79487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:40.500415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:40.578664 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:40.867185 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a18531-4d32-48bf-abc9-1bd59d573e39 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:40.869679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d7a18531-4d32-48bf-abc9-1bd59d573e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:40.984221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a18531-4d32-48bf-abc9-1bd59d573e39 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:41.005245 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:41.059612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2acb88-6fb5-4294-8aee-6ef687db2016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:41.060210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2acb88-6fb5-4294-8aee-6ef687db2016 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:41.060483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2acb88-6fb5-4294-8aee-6ef687db2016 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:41.060745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2acb88-6fb5-4294-8aee-6ef687db2016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:41.062353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9d2acb88-6fb5-4294-8aee-6ef687db2016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:41.180962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2acb88-6fb5-4294-8aee-6ef687db2016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:41.182422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2acb88-6fb5-4294-8aee-6ef687db2016 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:41.200841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 724/1452] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:43.670963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9b516-944c-4105-91f7-e8f5b3803347 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:43.671427 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9b516-944c-4105-91f7-e8f5b3803347 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:43.671527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9b516-944c-4105-91f7-e8f5b3803347 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:43.673598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:43.679186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd911ad7-aa80-4f22-8a61-18bba6febdf9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:43.679619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd911ad7-aa80-4f22-8a61-18bba6febdf9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:43.679738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd911ad7-aa80-4f22-8a61-18bba6febdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:43.746240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:43.749115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-dd911ad7-aa80-4f22-8a61-18bba6febdf9 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:43.750786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-dd911ad7-aa80-4f22-8a61-18bba6febdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:43.847346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:44.155166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9b516-944c-4105-91f7-e8f5b3803347 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:44.156711 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-19c9b516-944c-4105-91f7-e8f5b3803347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:44.267453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9b516-944c-4105-91f7-e8f5b3803347 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:44.318439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:44.344555 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd087a32-f7a1-4a24-9c79-5fcf1ed22a8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:44.345184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd087a32-f7a1-4a24-9c79-5fcf1ed22a8c service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:44.345466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd087a32-f7a1-4a24-9c79-5fcf1ed22a8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:44.345738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd087a32-f7a1-4a24-9c79-5fcf1ed22a8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:44.347331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bd087a32-f7a1-4a24-9c79-5fcf1ed22a8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:44.475108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd087a32-f7a1-4a24-9c79-5fcf1ed22a8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:44.476450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd087a32-f7a1-4a24-9c79-5fcf1ed22a8c service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:44.495054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 730/1455] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:46.785903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fbfd8-1b0c-4a8b-ad07-abda86a39cfe None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:46.786776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fbfd8-1b0c-4a8b-ad07-abda86a39cfe None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:46.786776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fbfd8-1b0c-4a8b-ad07-abda86a39cfe None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:46.789994 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:22:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:46.796737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e560f42-f54c-4884-83ab-83d002a3683a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:46.797217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e560f42-f54c-4884-83ab-83d002a3683a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:46.797328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e560f42-f54c-4884-83ab-83d002a3683a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:46.864893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9e560f42-f54c-4884-83ab-83d002a3683a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:46.866117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9e560f42-f54c-4884-83ab-83d002a3683a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:46.898733 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:46.969449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:47.250791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fbfd8-1b0c-4a8b-ad07-abda86a39cfe None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:47.252326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-873fbfd8-1b0c-4a8b-ad07-abda86a39cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:47.380836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fbfd8-1b0c-4a8b-ad07-abda86a39cfe None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:47.411159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:47.458892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98a5ff-0998-4130-b43f-8d60e7389e7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:47.459497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98a5ff-0998-4130-b43f-8d60e7389e7d service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:47.459769 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98a5ff-0998-4130-b43f-8d60e7389e7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:47.460039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98a5ff-0998-4130-b43f-8d60e7389e7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:47.461661 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bd98a5ff-0998-4130-b43f-8d60e7389e7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:47.589937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98a5ff-0998-4130-b43f-8d60e7389e7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:47.591368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98a5ff-0998-4130-b43f-8d60e7389e7d service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:47.730026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 727/1458] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:49.940152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5b9fa-3205-4606-a3ff-59f2dbb5d956 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:49.940723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5b9fa-3205-4606-a3ff-59f2dbb5d956 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:49.940723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5b9fa-3205-4606-a3ff-59f2dbb5d956 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:49.942109 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:49.947714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1160f49-31e7-4ecf-9ad9-ee942f1fcfa7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:49.948144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1160f49-31e7-4ecf-9ad9-ee942f1fcfa7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:49.948256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1160f49-31e7-4ecf-9ad9-ee942f1fcfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:50.018501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e1160f49-31e7-4ecf-9ad9-ee942f1fcfa7 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:50.019709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e1160f49-31e7-4ecf-9ad9-ee942f1fcfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:50.028097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:50.118577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:50.418525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5b9fa-3205-4606-a3ff-59f2dbb5d956 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:50.420075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4de5b9fa-3205-4606-a3ff-59f2dbb5d956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:50.527790 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5b9fa-3205-4606-a3ff-59f2dbb5d956 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:50.585256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:50.607224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c426cc-e928-4db0-9a9d-b2d1029b7fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:50.607839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c426cc-e928-4db0-9a9d-b2d1029b7fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:50.608114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c426cc-e928-4db0-9a9d-b2d1029b7fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:50.608746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c426cc-e928-4db0-9a9d-b2d1029b7fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:50.610375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a8c426cc-e928-4db0-9a9d-b2d1029b7fe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:50.727527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c426cc-e928-4db0-9a9d-b2d1029b7fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:50.728965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c426cc-e928-4db0-9a9d-b2d1029b7fe4 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:50.747884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 733/1461] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:52.917885 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069b1dd-f466-4f47-a1ff-840d43315368 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:52.918496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069b1dd-f466-4f47-a1ff-840d43315368 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:52.918496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069b1dd-f466-4f47-a1ff-840d43315368 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:52.919845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:52.925341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7821dd-92dc-4be3-b2ed-d523461f3447 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:52.925794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7821dd-92dc-4be3-b2ed-d523461f3447 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:52.925904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7821dd-92dc-4be3-b2ed-d523461f3447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:52.993262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7d7821dd-92dc-4be3-b2ed-d523461f3447 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:52.994436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7d7821dd-92dc-4be3-b2ed-d523461f3447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:53.037074 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:53.097051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:53.375246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069b1dd-f466-4f47-a1ff-840d43315368 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:53.376837 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2069b1dd-f466-4f47-a1ff-840d43315368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:53.492597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069b1dd-f466-4f47-a1ff-840d43315368 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:53.532690 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:53.575890 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1e71d-4124-4011-be0b-908a52834d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:53.576510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1e71d-4124-4011-be0b-908a52834d3b service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:53.576786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1e71d-4124-4011-be0b-908a52834d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:53.577059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1e71d-4124-4011-be0b-908a52834d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:53.578707 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7dd1e71d-4124-4011-be0b-908a52834d3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:53.694416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1e71d-4124-4011-be0b-908a52834d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:53.695858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1e71d-4124-4011-be0b-908a52834d3b service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:53.714209 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 730/1464] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:55.855311 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb68d09-a1e9-4572-9ce1-659abbbdb922 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:55.856231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb68d09-a1e9-4572-9ce1-659abbbdb922 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:55.856231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb68d09-a1e9-4572-9ce1-659abbbdb922 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:55.858779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:22:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:55.865137 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3418d6-65d8-4ceb-8fc5-9bce028cef77 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:55.865940 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3418d6-65d8-4ceb-8fc5-9bce028cef77 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:55.866261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3418d6-65d8-4ceb-8fc5-9bce028cef77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:55.942648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-3d3418d6-65d8-4ceb-8fc5-9bce028cef77 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:55.943788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3d3418d6-65d8-4ceb-8fc5-9bce028cef77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:55.963380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:56.051842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:56.338191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb68d09-a1e9-4572-9ce1-659abbbdb922 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:56.340178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4bb68d09-a1e9-4572-9ce1-659abbbdb922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:56.449955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb68d09-a1e9-4572-9ce1-659abbbdb922 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:56.523706 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:56.530850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd241c-4877-4930-920e-60229b1d484c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:56.531454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd241c-4877-4930-920e-60229b1d484c service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:56.531728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd241c-4877-4930-920e-60229b1d484c service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:56.531998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd241c-4877-4930-920e-60229b1d484c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:56.533638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-17fd241c-4877-4930-920e-60229b1d484c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:56.647854 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd241c-4877-4930-920e-60229b1d484c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:56.649239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd241c-4877-4930-920e-60229b1d484c service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:56.668211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 736/1467] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:58.806589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dcb43-6292-444a-b4a5-1ddcb9ce19da None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:58.807471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dcb43-6292-444a-b4a5-1ddcb9ce19da None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:58.807471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dcb43-6292-444a-b4a5-1ddcb9ce19da None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:58.809534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:22:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:22:58.815847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc9b50-e1f0-4069-99e5-402558d15d72 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:58.816304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc9b50-e1f0-4069-99e5-402558d15d72 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:58.816416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc9b50-e1f0-4069-99e5-402558d15d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:58.887891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-70bc9b50-e1f0-4069-99e5-402558d15d72 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:22:58.889116 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-70bc9b50-e1f0-4069-99e5-402558d15d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:58.939703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:58.989889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:22:59.273618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dcb43-6292-444a-b4a5-1ddcb9ce19da None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:22:59.275147 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-647dcb43-6292-444a-b4a5-1ddcb9ce19da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:59.391046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dcb43-6292-444a-b4a5-1ddcb9ce19da None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:22:59.446705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:59.474217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcd6f5-ee6c-41d2-a805-cd5e4a95116d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:22:59.474821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcd6f5-ee6c-41d2-a805-cd5e4a95116d service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:22:59.475160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcd6f5-ee6c-41d2-a805-cd5e4a95116d service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:22:59.475340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcd6f5-ee6c-41d2-a805-cd5e4a95116d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:22:59.476951 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4bbcd6f5-ee6c-41d2-a805-cd5e4a95116d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:22:59.601504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcd6f5-ee6c-41d2-a805-cd5e4a95116d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:22:59.603199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcd6f5-ee6c-41d2-a805-cd5e4a95116d service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:22:59.627558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 733/1470] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:23:08.988415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=22 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:09.624599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=66 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:19.502166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48aaab-fec6-4d1f-8c69-d6db4d59af3b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:19.502736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48aaab-fec6-4d1f-8c69-d6db4d59af3b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:19.502736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48aaab-fec6-4d1f-8c69-d6db4d59af3b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:19.503950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 738/1471] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:23:19.510455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ae831-9a66-42d9-a4af-82cff63c0102 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:19.511204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ae831-9a66-42d9-a4af-82cff63c0102 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:19.511361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ae831-9a66-42d9-a4af-82cff63c0102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:19.585362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5f6ae831-9a66-42d9-a4af-82cff63c0102 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:23:19.586573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5f6ae831-9a66-42d9-a4af-82cff63c0102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:19.688208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 734/1472] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:23:19.985459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2651aae-968a-45ec-a7c7-7c19522678b5 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:19.985730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2651aae-968a-45ec-a7c7-7c19522678b5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:19.985994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2651aae-968a-45ec-a7c7-7c19522678b5 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:19.987337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 739/1473] 10.222.0.60 () {54 vars in 962 bytes} [Sat Sep 17 04:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:23:19.993152 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce50e2-8b22-4912-90f5-b2ef0d5606c7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:19.993553 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce50e2-8b22-4912-90f5-b2ef0d5606c7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:19.993884 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce50e2-8b22-4912-90f5-b2ef0d5606c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:20.076245 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a5ce50e2-8b22-4912-90f5-b2ef0d5606c7 None None] MFA Rules not processed for user `acf8a0ae03c94aa78cc2ab0e4e1c87b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:23:20.077503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a5ce50e2-8b22-4912-90f5-b2ef0d5606c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:20.101483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:20.165624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 735/1474] 10.222.0.60 () {58 vars in 1059 bytes} [Sat Sep 17 04:23:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:23:20.173618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2651aae-968a-45ec-a7c7-7c19522678b5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:23:20.175135 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e2651aae-968a-45ec-a7c7-7c19522678b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:20.281712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2651aae-968a-45ec-a7c7-7c19522678b5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:23:20.358965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f85a31-1069-43c4-bee9-db388c4f5505 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf8a0ae03c94aa78cc2ab0e4e1c87b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:23:20.359578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f85a31-1069-43c4-bee9-db388c4f5505 service glance] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:20.359849 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f85a31-1069-43c4-bee9-db388c4f5505 service glance] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:20.360118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f85a31-1069-43c4-bee9-db388c4f5505 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:20.361735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f5f85a31-1069-43c4-bee9-db388c4f5505 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:20.424099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:20.480862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f85a31-1069-43c4-bee9-db388c4f5505 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:23:20.482336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f85a31-1069-43c4-bee9-db388c4f5505 service glance] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:23:20.501973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 740/1475] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:23:22.766972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c6662-9670-4695-83a2-cb2edee71765 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:22.767919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c6662-9670-4695-83a2-cb2edee71765 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:22.767919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c6662-9670-4695-83a2-cb2edee71765 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:22.770110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 736/1476] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:23:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:23:22.775696 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab2b01-64b2-4562-b842-7322779efb44 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:22.776155 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab2b01-64b2-4562-b842-7322779efb44 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:22.776263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab2b01-64b2-4562-b842-7322779efb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:22.845017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f2ab2b01-64b2-4562-b842-7322779efb44 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:23:22.846258 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f2ab2b01-64b2-4562-b842-7322779efb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:22.944486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 741/1477] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:23:23.215332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c6662-9670-4695-83a2-cb2edee71765 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:23:23.216806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b26c6662-9670-4695-83a2-cb2edee71765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:23.361460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c6662-9670-4695-83a2-cb2edee71765 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:23:23.411691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:23.455602 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae464028-307d-45d4-b418-a1dbec19276b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:23:23.456201 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae464028-307d-45d4-b418-a1dbec19276b admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:23.456468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae464028-307d-45d4-b418-a1dbec19276b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:23.456727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae464028-307d-45d4-b418-a1dbec19276b admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:23.457387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae464028-307d-45d4-b418-a1dbec19276b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:23:23.459125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-ae464028-307d-45d4-b418-a1dbec19276b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:23:23.459125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 17 04:23:23.459125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: Sep 17 04:23:23.460216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae464028-307d-45d4-b418-a1dbec19276b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:23:23.479198 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 737/1478] 10.222.0.60 () {56 vars in 1108 bytes} [Sat Sep 17 04:23:23 2022] GET /identity/v3/projects => generated 2553 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:23:25.637708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c55eb-3ff0-41c9-bc22-459108ab5118 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:25.638366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c55eb-3ff0-41c9-bc22-459108ab5118 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:25.638366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c55eb-3ff0-41c9-bc22-459108ab5118 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:25.639650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 742/1479] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:23:25.646173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508bece-08fb-4b2b-b516-498852ba47b5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:25.646890 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508bece-08fb-4b2b-b516-498852ba47b5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:25.646999 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508bece-08fb-4b2b-b516-498852ba47b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:25.719658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d508bece-08fb-4b2b-b516-498852ba47b5 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:23:25.720846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d508bece-08fb-4b2b-b516-498852ba47b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:25.817825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 738/1480] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:23:26.121230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c55eb-3ff0-41c9-bc22-459108ab5118 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:23:26.123057 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-537c55eb-3ff0-41c9-bc22-459108ab5118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:26.134121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:26.280062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c55eb-3ff0-41c9-bc22-459108ab5118 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:23:26.370891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:26.403744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d9d87-bd9a-4d52-a07f-9039f4af86cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:23:26.404350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d9d87-bd9a-4d52-a07f-9039f4af86cc service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:26.404625 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d9d87-bd9a-4d52-a07f-9039f4af86cc service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:26.404897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d9d87-bd9a-4d52-a07f-9039f4af86cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:26.406556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ba0d9d87-bd9a-4d52-a07f-9039f4af86cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:26.501858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d9d87-bd9a-4d52-a07f-9039f4af86cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:23:26.503437 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d9d87-bd9a-4d52-a07f-9039f4af86cc service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:23:26.521203 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 743/1481] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:23:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:23:28.858728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc7005-598a-4349-95b0-2b983ba4e995 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:28.859651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc7005-598a-4349-95b0-2b983ba4e995 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:28.859651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc7005-598a-4349-95b0-2b983ba4e995 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:28.861821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 739/1482] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:23:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:23:28.870301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4dbe-64f2-4fe6-89d9-e0d7bac55a15 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:28.870752 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4dbe-64f2-4fe6-89d9-e0d7bac55a15 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:28.870867 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4dbe-64f2-4fe6-89d9-e0d7bac55a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:28.938198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7a9b4dbe-64f2-4fe6-89d9-e0d7bac55a15 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:23:28.939325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7a9b4dbe-64f2-4fe6-89d9-e0d7bac55a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:28.987770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:29.039838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 744/1483] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:23:29.327632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc7005-598a-4349-95b0-2b983ba4e995 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:23:29.330079 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d2fc7005-598a-4349-95b0-2b983ba4e995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:29.360020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:29.458213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc7005-598a-4349-95b0-2b983ba4e995 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:23:29.536746 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfa29b-e01c-425f-9d96-90f1ee7beda8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:23:29.537380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfa29b-e01c-425f-9d96-90f1ee7beda8 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:29.537654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfa29b-e01c-425f-9d96-90f1ee7beda8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:29.537919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfa29b-e01c-425f-9d96-90f1ee7beda8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:29.539514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6adfa29b-e01c-425f-9d96-90f1ee7beda8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:29.616438 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:29.652993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfa29b-e01c-425f-9d96-90f1ee7beda8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:23:29.654523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfa29b-e01c-425f-9d96-90f1ee7beda8 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:23:29.672970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 740/1484] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:23:31.874520 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e32af0-1d30-4c87-85f1-266b901436ed None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:31.875121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e32af0-1d30-4c87-85f1-266b901436ed None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:31.875121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e32af0-1d30-4c87-85f1-266b901436ed None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:31.877153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 745/1485] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:23:31.885501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a0e96-2c81-4934-9d9f-7e88ef4db215 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:31.885965 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a0e96-2c81-4934-9d9f-7e88ef4db215 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:31.886094 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a0e96-2c81-4934-9d9f-7e88ef4db215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:31.957215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-704a0e96-2c81-4934-9d9f-7e88ef4db215 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:23:31.958296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-704a0e96-2c81-4934-9d9f-7e88ef4db215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:32.069491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 741/1486] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:23:32.354771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e32af0-1d30-4c87-85f1-266b901436ed None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:23:32.357114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e8e32af0-1d30-4c87-85f1-266b901436ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:32.474084 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e32af0-1d30-4c87-85f1-266b901436ed None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:23:32.553841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:32.555420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b30c1-d9f7-44fc-81ed-082ba75a8a16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:23:32.556236 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b30c1-d9f7-44fc-81ed-082ba75a8a16 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:32.556556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b30c1-d9f7-44fc-81ed-082ba75a8a16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:32.556834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b30c1-d9f7-44fc-81ed-082ba75a8a16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:32.559332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-378b30c1-d9f7-44fc-81ed-082ba75a8a16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:32.674148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b30c1-d9f7-44fc-81ed-082ba75a8a16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:23:32.675606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b30c1-d9f7-44fc-81ed-082ba75a8a16 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:23:32.693851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 746/1487] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:23:35.045692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725309d6-ab67-4649-a8c4-3fb5cf7781e3 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:35.046649 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725309d6-ab67-4649-a8c4-3fb5cf7781e3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:35.046649 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725309d6-ab67-4649-a8c4-3fb5cf7781e3 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:35.048750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 742/1488] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:23:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:23:35.055166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ec6ea-41a9-43b8-8f69-da9f255a851f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:35.055898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ec6ea-41a9-43b8-8f69-da9f255a851f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:35.056057 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ec6ea-41a9-43b8-8f69-da9f255a851f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:35.126485 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cf7ec6ea-41a9-43b8-8f69-da9f255a851f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:23:35.127692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cf7ec6ea-41a9-43b8-8f69-da9f255a851f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:35.184400 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:35.229542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 747/1489] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:23:35.523992 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725309d6-ab67-4649-a8c4-3fb5cf7781e3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:23:35.526524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-725309d6-ab67-4649-a8c4-3fb5cf7781e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:35.571892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:35.654362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725309d6-ab67-4649-a8c4-3fb5cf7781e3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:23:35.732154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43884f39-4404-4bb6-b92c-7221da46de04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:23:35.733110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43884f39-4404-4bb6-b92c-7221da46de04 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:35.733403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43884f39-4404-4bb6-b92c-7221da46de04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:35.733673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43884f39-4404-4bb6-b92c-7221da46de04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:35.735261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-43884f39-4404-4bb6-b92c-7221da46de04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:35.819394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:35.852025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43884f39-4404-4bb6-b92c-7221da46de04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:23:35.853522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43884f39-4404-4bb6-b92c-7221da46de04 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:23:35.872124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 743/1490] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:23:38.221071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f18e15-6148-4092-8fc9-66f773395ab8 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:38.221706 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f18e15-6148-4092-8fc9-66f773395ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:38.221706 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f18e15-6148-4092-8fc9-66f773395ab8 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:38.223005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 748/1491] 10.222.0.60 () {54 vars in 927 bytes} [Sat Sep 17 04:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:23:38.229557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215db03-026d-4b9e-8ede-009cb3a66880 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:38.230023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215db03-026d-4b9e-8ede-009cb3a66880 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:38.230130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215db03-026d-4b9e-8ede-009cb3a66880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:38.303877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2215db03-026d-4b9e-8ede-009cb3a66880 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:23:38.305120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2215db03-026d-4b9e-8ede-009cb3a66880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:38.409042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 744/1492] 10.222.0.60 () {58 vars in 1024 bytes} [Sat Sep 17 04:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:23:38.699077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f18e15-6148-4092-8fc9-66f773395ab8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:23:38.701251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-44f18e15-6148-4092-8fc9-66f773395ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:38.811001 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f18e15-6148-4092-8fc9-66f773395ab8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:23:38.888244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa3ccb-4243-4dc6-b558-3572b9f43c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:23:38.888850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa3ccb-4243-4dc6-b558-3572b9f43c07 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:23:38.889114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa3ccb-4243-4dc6-b558-3572b9f43c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:23:38.889422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa3ccb-4243-4dc6-b558-3572b9f43c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:23:38.891006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1bfa3ccb-4243-4dc6-b558-3572b9f43c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:23:38.899642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:39.013826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa3ccb-4243-4dc6-b558-3572b9f43c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:23:39.015213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa3ccb-4243-4dc6-b558-3572b9f43c07 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:23:39.033978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 749/1493] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:23:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:23:48.407285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=82 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:49.031314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=50 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:24:51.481011 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557029e8-cbf1-4df7-99fe-3afddc9faa15 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:51.482002 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557029e8-cbf1-4df7-99fe-3afddc9faa15 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:51.482002 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557029e8-cbf1-4df7-99fe-3afddc9faa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:51.576276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-557029e8-cbf1-4df7-99fe-3afddc9faa15 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:24:51.577430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-557029e8-cbf1-4df7-99fe-3afddc9faa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:51.678992 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 745/1494] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:24:51.687192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67224a80-1cb4-4dd0-ade4-823167226fb3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:51.687703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67224a80-1cb4-4dd0-ade4-823167226fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:51.687703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67224a80-1cb4-4dd0-ade4-823167226fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:51.772835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-67224a80-1cb4-4dd0-ade4-823167226fb3 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:24:51.774071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-67224a80-1cb4-4dd0-ade4-823167226fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:51.872391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 750/1495] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:24:51.876982 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557029e8-cbf1-4df7-99fe-3afddc9faa15 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:24:51.878323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-557029e8-cbf1-4df7-99fe-3afddc9faa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:51.889869 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557029e8-cbf1-4df7-99fe-3afddc9faa15 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:24:51.953233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:24:51.987797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62d8c1-01b2-41aa-9ceb-5d6678c6a6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:24:51.988398 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62d8c1-01b2-41aa-9ceb-5d6678c6a6cb admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:51.988667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62d8c1-01b2-41aa-9ceb-5d6678c6a6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:51.988933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62d8c1-01b2-41aa-9ceb-5d6678c6a6cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:51.989598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62d8c1-01b2-41aa-9ceb-5d6678c6a6cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:24:51.989888 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62d8c1-01b2-41aa-9ceb-5d6678c6a6cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:24:51.990886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-bf62d8c1-01b2-41aa-9ceb-5d6678c6a6cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:24:51.990886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 17 04:24:51.990886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: Sep 17 04:24:51.991772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62d8c1-01b2-41aa-9ceb-5d6678c6a6cb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:24:52.002716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 746/1496] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:24:52.011202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67224a80-1cb4-4dd0-ade4-823167226fb3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:24:52.020383 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67224a80-1cb4-4dd0-ade4-823167226fb3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:24:52.081982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:24:52.118310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b4972-615d-431e-90b7-98ecb1c82093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:24:52.118907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b4972-615d-431e-90b7-98ecb1c82093 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:52.119178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b4972-615d-431e-90b7-98ecb1c82093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:52.119443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b4972-615d-431e-90b7-98ecb1c82093 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:52.120068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b4972-615d-431e-90b7-98ecb1c82093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:24:52.121838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-680b4972-615d-431e-90b7-98ecb1c82093 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:24:52.121838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 17 04:24:52.121838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: Sep 17 04:24:52.123074 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b4972-615d-431e-90b7-98ecb1c82093 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:24:52.180330 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 751/1497] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:24:52 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:24:52.184766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62d8c1-01b2-41aa-9ceb-5d6678c6a6cb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:24:52.186321 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bf62d8c1-01b2-41aa-9ceb-5d6678c6a6cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:52.220821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62d8c1-01b2-41aa-9ceb-5d6678c6a6cb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:24:52.318407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f080be-102d-4887-920f-1387c653e259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:24:52.319019 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f080be-102d-4887-920f-1387c653e259 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:52.319297 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f080be-102d-4887-920f-1387c653e259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:52.319597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f080be-102d-4887-920f-1387c653e259 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:52.320231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f080be-102d-4887-920f-1387c653e259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:24:52.321272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-96f080be-102d-4887-920f-1387c653e259 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:24:52.321272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 17 04:24:52.321272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: Sep 17 04:24:52.322479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f080be-102d-4887-920f-1387c653e259 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:24:52.369991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 747/1498] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:24:52 2022] POST /identity/v3/users => generated 290 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:24:52.374487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b4972-615d-431e-90b7-98ecb1c82093 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:24:52.375950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-680b4972-615d-431e-90b7-98ecb1c82093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:52.442471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b4972-615d-431e-90b7-98ecb1c82093 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:24:52.528104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:24:52.539634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d0ef5-36ab-458f-8690-0897fd037e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:24:52.540252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d0ef5-36ab-458f-8690-0897fd037e8f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:52.540542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d0ef5-36ab-458f-8690-0897fd037e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:52.540899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d0ef5-36ab-458f-8690-0897fd037e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:52.541852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d0ef5-36ab-458f-8690-0897fd037e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:24:52.542873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-0c4d0ef5-36ab-458f-8690-0897fd037e8f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:24:52.542873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 17 04:24:52.542873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: Sep 17 04:24:52.543763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d0ef5-36ab-458f-8690-0897fd037e8f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:24:52.556743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 752/1499] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:24:52.561501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f080be-102d-4887-920f-1387c653e259 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:24:52.563003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-96f080be-102d-4887-920f-1387c653e259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:52.574652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f080be-102d-4887-920f-1387c653e259 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:24:52.601135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:24:52.675977 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f55f6-75f9-4c75-8e98-eb744f2d69db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:24:52.676595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f55f6-75f9-4c75-8e98-eb744f2d69db admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:52.676872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f55f6-75f9-4c75-8e98-eb744f2d69db admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:52.677200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f55f6-75f9-4c75-8e98-eb744f2d69db admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:52.677854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f55f6-75f9-4c75-8e98-eb744f2d69db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:24:52.678873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-833f55f6-75f9-4c75-8e98-eb744f2d69db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:24:52.678873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 17 04:24:52.678873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: Sep 17 04:24:52.679762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f55f6-75f9-4c75-8e98-eb744f2d69db admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:24:52.692708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 748/1500] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:24:52.697191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d0ef5-36ab-458f-8690-0897fd037e8f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:24:52.698724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0c4d0ef5-36ab-458f-8690-0897fd037e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:52.736509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d0ef5-36ab-458f-8690-0897fd037e8f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:24:52.834951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c853c5-8a27-4123-bb77-0623b71e8f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:24:52.835610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c853c5-8a27-4123-bb77-0623b71e8f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:52.835883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c853c5-8a27-4123-bb77-0623b71e8f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:52.836163 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c853c5-8a27-4123-bb77-0623b71e8f50 admin admin] PATH_INFO: `/v3/projects/a8d9b1c563d94881833202368070d8c2/users/28b04f08204c4c3a9e22e0eec708100a/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:52.860967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c853c5-8a27-4123-bb77-0623b71e8f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d9b1c563d94881833202368070d8c2, user_id=28b04f08204c4c3a9e22e0eec708100a, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:24:52.862046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-12c853c5-8a27-4123-bb77-0623b71e8f50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:24:52.862046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 17 04:24:52.862046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: Sep 17 04:24:52.863975 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c853c5-8a27-4123-bb77-0623b71e8f50 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:24:52.893132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 753/1501] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:24:52 2022] PUT /identity/v3/projects/a8d9b1c563d94881833202368070d8c2/users/28b04f08204c4c3a9e22e0eec708100a/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:24:52.905799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678f672-4289-4bc9-bc2c-2b5ad4d28480 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:52.906086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678f672-4289-4bc9-bc2c-2b5ad4d28480 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:52.906356 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678f672-4289-4bc9-bc2c-2b5ad4d28480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:52.963424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7678f672-4289-4bc9-bc2c-2b5ad4d28480 None None] MFA Rules not processed for user `28b04f08204c4c3a9e22e0eec708100a`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:24:52.964547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7678f672-4289-4bc9-bc2c-2b5ad4d28480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:53.041992 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 749/1502] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:24:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:24:53.055929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce938a7d-aba0-44c4-a6a1-3366721ff3fe None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:53.056214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce938a7d-aba0-44c4-a6a1-3366721ff3fe None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:53.056486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce938a7d-aba0-44c4-a6a1-3366721ff3fe None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:53.057865 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 754/1503] 10.222.0.60 () {54 vars in 964 bytes} [Sat Sep 17 04:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:24:53.062745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601b07d-d978-42c3-ab1b-b6cb3bc67df5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:53.063025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601b07d-d978-42c3-ab1b-b6cb3bc67df5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:53.063298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601b07d-d978-42c3-ab1b-b6cb3bc67df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:53.096021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:24:53.141603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e601b07d-d978-42c3-ab1b-b6cb3bc67df5 None None] MFA Rules not processed for user `6adf758d5fb9430189ee9057b4b7ea97`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:24:53.142799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e601b07d-d978-42c3-ab1b-b6cb3bc67df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:53.220201 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 750/1504] 10.222.0.60 () {58 vars in 1061 bytes} [Sat Sep 17 04:24:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:24:53.229806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce938a7d-aba0-44c4-a6a1-3366721ff3fe None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:24:53.231318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ce938a7d-aba0-44c4-a6a1-3366721ff3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:53.297384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:24:53.354496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce938a7d-aba0-44c4-a6a1-3366721ff3fe None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:24:53.434121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58cfc9-0225-4ca2-9f31-a4a933859bb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:24:53.434752 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58cfc9-0225-4ca2-9f31-a4a933859bb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:53.435025 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58cfc9-0225-4ca2-9f31-a4a933859bb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:53.435281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58cfc9-0225-4ca2-9f31-a4a933859bb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:53.436908 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2f58cfc9-0225-4ca2-9f31-a4a933859bb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:53.557429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58cfc9-0225-4ca2-9f31-a4a933859bb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:24:53.558866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58cfc9-0225-4ca2-9f31-a4a933859bb0 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:24:53.562320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:24:53.587413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 755/1505] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:24:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:24:53.633757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568e08c-1878-444e-930e-6e7d9f02ca66 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:53.634195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568e08c-1878-444e-930e-6e7d9f02ca66 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:53.634411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568e08c-1878-444e-930e-6e7d9f02ca66 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:53.635697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 751/1506] 10.222.0.60 () {52 vars in 830 bytes} [Sat Sep 17 04:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:24:53.639527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58cfc9-0225-4ca2-9f31-a4a933859bb0 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:24:53.640816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2f58cfc9-0225-4ca2-9f31-a4a933859bb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:53.746669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58cfc9-0225-4ca2-9f31-a4a933859bb0 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:24:53.841805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:24:53.842353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:53.842623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:53.842881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] PATH_INFO: `/v3/users/28b04f08204c4c3a9e22e0eec708100a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:53.856060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b04f08204c4c3a9e22e0eec708100a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:24:53.857504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:24:53.857504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 17 04:24:53.857504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: Sep 17 04:24:53.858704 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:24:53.869268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:24:53.935842 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b04f08204c4c3a9e22e0eec708100a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:24:53.946855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b04f08204c4c3a9e22e0eec708100a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:24:53.947335 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b04f08204c4c3a9e22e0eec708100a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:24:53.967659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b04f08204c4c3a9e22e0eec708100a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:24:54.005628 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b04f08204c4c3a9e22e0eec708100a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:24:54.013897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 756/1507] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:24:53 2022] DELETE /identity/v3/users/28b04f08204c4c3a9e22e0eec708100a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:24:54.019751 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba7893-8f09-47b2-8d44-6b474c693041 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:54.020234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba7893-8f09-47b2-8d44-6b474c693041 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:54.020613 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba7893-8f09-47b2-8d44-6b474c693041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:54.092229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a5ba7893-8f09-47b2-8d44-6b474c693041 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:24:54.093380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a5ba7893-8f09-47b2-8d44-6b474c693041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:54.171893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:24:54.185787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 752/1508] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:24:54.196467 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:24:54.197802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:54.302563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e78fda-7845-4069-9f4b-379907932ce6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:24:54.369633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:24:54.390906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58654ac1-f59d-48ee-a4d8-c7c4e154857f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:24:54.391523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58654ac1-f59d-48ee-a4d8-c7c4e154857f service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:54.391795 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58654ac1-f59d-48ee-a4d8-c7c4e154857f service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:54.392063 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58654ac1-f59d-48ee-a4d8-c7c4e154857f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:54.393721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-58654ac1-f59d-48ee-a4d8-c7c4e154857f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:54.504494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58654ac1-f59d-48ee-a4d8-c7c4e154857f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:24:54.505988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58654ac1-f59d-48ee-a4d8-c7c4e154857f service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:24:54.524597 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 757/1509] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:24:54.892955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba7893-8f09-47b2-8d44-6b474c693041 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:24:54.894344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a5ba7893-8f09-47b2-8d44-6b474c693041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:24:54.905720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba7893-8f09-47b2-8d44-6b474c693041 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:24:55.013522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:24:55.014178 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:24:55.014444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:24:55.014715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] PATH_INFO: `/v3/projects/a8d9b1c563d94881833202368070d8c2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:24:55.025419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d9b1c563d94881833202368070d8c2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:24:55.026440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:24:55.026440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 17 04:24:55.026440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: Sep 17 04:24:55.027635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:24:55.092621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d9b1c563d94881833202368070d8c2', 'request_id': 'req-099d5735-39aa-45b5-875b-78b2e63b9cda'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:24:55.103642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d9b1c563d94881833202368070d8c2', 'request_id': 'req-099d5735-39aa-45b5-875b-78b2e63b9cda'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:24:55.110850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d9b1c563d94881833202368070d8c2', 'request_id': 'req-099d5735-39aa-45b5-875b-78b2e63b9cda'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:24:55.142623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] The token cache is being invalidate because project a8d9b1c563d94881833202368070d8c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:24:55.142799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:24:55.143246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-099d5735-39aa-45b5-875b-78b2e63b9cda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:24:55.144619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 753/1510] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:24:54 2022] DELETE /identity/v3/projects/a8d9b1c563d94881833202368070d8c2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:25:04.522332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=62 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:25:05.140446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:26:23.534892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b6dbb-3dbb-4026-8f39-78dc3344c748 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:23.535521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b6dbb-3dbb-4026-8f39-78dc3344c748 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:23.535521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b6dbb-3dbb-4026-8f39-78dc3344c748 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:23.536889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 04:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:26:23.542038 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f85174-c5a6-469a-a11a-306d6df7baf5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:23.542503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f85174-c5a6-469a-a11a-306d6df7baf5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:23.542612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f85174-c5a6-469a-a11a-306d6df7baf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:23.627520 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a9f85174-c5a6-469a-a11a-306d6df7baf5 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:26:23.628613 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a9f85174-c5a6-469a-a11a-306d6df7baf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:23.739753 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 04:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:26:23.898304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b6dbb-3dbb-4026-8f39-78dc3344c748 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:26:23.901879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0b5b6dbb-3dbb-4026-8f39-78dc3344c748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:24.038283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b6dbb-3dbb-4026-8f39-78dc3344c748 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:26:24.044275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66657cf5-290d-4d71-ad26-30ffd8009267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:26:24.044879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66657cf5-290d-4d71-ad26-30ffd8009267 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:24.045187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66657cf5-290d-4d71-ad26-30ffd8009267 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:24.045453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66657cf5-290d-4d71-ad26-30ffd8009267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:24.053947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66657cf5-290d-4d71-ad26-30ffd8009267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:26:24.055528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66657cf5-290d-4d71-ad26-30ffd8009267 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:26:24.064824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 759/1513] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:26:25.144734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf51de0-0a50-4d2d-8aca-f41d165504b0 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:25.145711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf51de0-0a50-4d2d-8aca-f41d165504b0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:25.145711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf51de0-0a50-4d2d-8aca-f41d165504b0 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:25.147823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 04:26:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:26:25.157314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d846a9d-1838-4193-9315-976f5e011553 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:25.157789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d846a9d-1838-4193-9315-976f5e011553 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:25.157895 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d846a9d-1838-4193-9315-976f5e011553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:25.177833 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5d846a9d-1838-4193-9315-976f5e011553 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:26:25.179005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5d846a9d-1838-4193-9315-976f5e011553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:25.187247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 04:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:26:25.349328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf51de0-0a50-4d2d-8aca-f41d165504b0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:26:25.365390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf51de0-0a50-4d2d-8aca-f41d165504b0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:26:25.371381 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30673ab3-c71a-4160-a574-dc235a65e356 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:26:25.371978 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30673ab3-c71a-4160-a574-dc235a65e356 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:25.372255 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30673ab3-c71a-4160-a574-dc235a65e356 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:25.372522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30673ab3-c71a-4160-a574-dc235a65e356 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:25.380761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30673ab3-c71a-4160-a574-dc235a65e356 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:26:25.381285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f55bb2-017b-453e-922b-0ac69f40f9f4 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:25.381570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f55bb2-017b-453e-922b-0ac69f40f9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:25.381842 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f55bb2-017b-453e-922b-0ac69f40f9f4 None None] PATH_INFO: `/v3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:25.382383 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30673ab3-c71a-4160-a574-dc235a65e356 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:26:25.383162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 761/1516] 10.222.0.66 () {54 vars in 925 bytes} [Sat Sep 17 04:26:25 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:26:25.389932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742e760-8595-4713-8568-16326f1b6348 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:25.390213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742e760-8595-4713-8568-16326f1b6348 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:25.390483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742e760-8595-4713-8568-16326f1b6348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:25.391702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 756/1517] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:26:25.455454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1742e760-8595-4713-8568-16326f1b6348 None None] MFA Rules not processed for user `e6d3b7887c13466ab8e3d9737f907c56`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:26:25.456698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1742e760-8595-4713-8568-16326f1b6348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:25.515908 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:26:25.536713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 762/1518] 10.222.0.66 () {58 vars in 1015 bytes} [Sat Sep 17 04:26:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:26:34.704496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e9b85-6577-47c5-a80b-02497776e227 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:34.704987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e9b85-6577-47c5-a80b-02497776e227 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:34.705115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e9b85-6577-47c5-a80b-02497776e227 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:34.706624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 757/1519] 10.222.0.60 () {54 vars in 961 bytes} [Sat Sep 17 04:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:26:34.711936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b1b11-a43a-4b75-8f4c-ad1be0e10634 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:34.712398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b1b11-a43a-4b75-8f4c-ad1be0e10634 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:34.712500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b1b11-a43a-4b75-8f4c-ad1be0e10634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:34.733107 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8d6b1b11-a43a-4b75-8f4c-ad1be0e10634 None None] MFA Rules not processed for user `e6d3b7887c13466ab8e3d9737f907c56`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:26:34.734393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8d6b1b11-a43a-4b75-8f4c-ad1be0e10634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:34.746100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 763/1520] 10.222.0.60 () {58 vars in 1058 bytes} [Sat Sep 17 04:26:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:26:34.754130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e9b85-6577-47c5-a80b-02497776e227 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:26:34.774606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e9b85-6577-47c5-a80b-02497776e227 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:26:34.780625 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6c0bf-4e01-4b65-a675-59a0404d6864 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:26:34.781281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6c0bf-4e01-4b65-a675-59a0404d6864 service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:34.781559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6c0bf-4e01-4b65-a675-59a0404d6864 service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:34.781849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6c0bf-4e01-4b65-a675-59a0404d6864 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:34.790525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6c0bf-4e01-4b65-a675-59a0404d6864 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:26:34.792085 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6c0bf-4e01-4b65-a675-59a0404d6864 service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:26:34.801720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 758/1521] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:26:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:26:36.880332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbb08d-3875-426f-a3b7-aa867703f4ae None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:36.880935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbb08d-3875-426f-a3b7-aa867703f4ae None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:36.880935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbb08d-3875-426f-a3b7-aa867703f4ae None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:36.882271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 04:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:26:36.888763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f40a4e-480f-40aa-9175-9b300ffcd9c4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:36.889266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f40a4e-480f-40aa-9175-9b300ffcd9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:36.889375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f40a4e-480f-40aa-9175-9b300ffcd9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:36.911952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-75f40a4e-480f-40aa-9175-9b300ffcd9c4 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:26:36.913200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-75f40a4e-480f-40aa-9175-9b300ffcd9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:36.922485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 04:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:26:37.083697 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbb08d-3875-426f-a3b7-aa867703f4ae None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:26:37.105497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbb08d-3875-426f-a3b7-aa867703f4ae None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:26:37.112043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e14ae-87c0-462c-adf1-f03dfb9fd469 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:26:37.112639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e14ae-87c0-462c-adf1-f03dfb9fd469 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:37.112922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e14ae-87c0-462c-adf1-f03dfb9fd469 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:37.113246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e14ae-87c0-462c-adf1-f03dfb9fd469 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:37.121576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e14ae-87c0-462c-adf1-f03dfb9fd469 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:26:37.123162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e14ae-87c0-462c-adf1-f03dfb9fd469 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:26:37.132560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 765/1524] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:26:38.221653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd2286-1c3e-4beb-b49f-36fcc84c2e82 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:38.222580 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd2286-1c3e-4beb-b49f-36fcc84c2e82 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:38.222580 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd2286-1c3e-4beb-b49f-36fcc84c2e82 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:38.224596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 04:26:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:26:38.232167 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0b88c-2af1-4147-9b9a-f860b6660e15 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:38.232845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0b88c-2af1-4147-9b9a-f860b6660e15 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:38.232992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0b88c-2af1-4147-9b9a-f860b6660e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:38.255712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d4e0b88c-2af1-4147-9b9a-f860b6660e15 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:26:38.256936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d4e0b88c-2af1-4147-9b9a-f860b6660e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:38.266255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 04:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:26:38.418490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd2286-1c3e-4beb-b49f-36fcc84c2e82 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:26:38.436800 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd2286-1c3e-4beb-b49f-36fcc84c2e82 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:26:38.443262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7a790-4c6b-4e88-9c85-33795981a61e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:26:38.443862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7a790-4c6b-4e88-9c85-33795981a61e service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:38.444137 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7a790-4c6b-4e88-9c85-33795981a61e service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:38.444407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7a790-4c6b-4e88-9c85-33795981a61e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:38.452655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7a790-4c6b-4e88-9c85-33795981a61e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:26:38.454256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7a790-4c6b-4e88-9c85-33795981a61e service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:26:38.464211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 761/1527] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:26:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:26:42.302367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05854e-fbc9-495b-b804-20ed7e3e6004 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:42.303397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05854e-fbc9-495b-b804-20ed7e3e6004 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:42.303397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05854e-fbc9-495b-b804-20ed7e3e6004 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:42.305412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:26:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:26:42.314725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb8f8f-9cc7-4781-91f1-cf798d6be5a3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:42.315182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb8f8f-9cc7-4781-91f1-cf798d6be5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:42.315291 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb8f8f-9cc7-4781-91f1-cf798d6be5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:42.336303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-33bb8f8f-9cc7-4781-91f1-cf798d6be5a3 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:26:42.337550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-33bb8f8f-9cc7-4781-91f1-cf798d6be5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:42.346798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:26:42.641054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05854e-fbc9-495b-b804-20ed7e3e6004 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:26:42.644269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ab05854e-fbc9-495b-b804-20ed7e3e6004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:42.726500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05854e-fbc9-495b-b804-20ed7e3e6004 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:26:42.731047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8295d3-394f-4c02-bd7f-23e76217c9c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:26:42.731650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8295d3-394f-4c02-bd7f-23e76217c9c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:42.731930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8295d3-394f-4c02-bd7f-23e76217c9c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:42.732197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8295d3-394f-4c02-bd7f-23e76217c9c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:42.741433 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8295d3-394f-4c02-bd7f-23e76217c9c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:26:42.742864 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8295d3-394f-4c02-bd7f-23e76217c9c9 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:26:42.752164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 768/1530] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:26:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:26:44.882022 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf4237-910b-4232-959b-3acf4e21feff None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:44.882624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf4237-910b-4232-959b-3acf4e21feff None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:44.882624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf4237-910b-4232-959b-3acf4e21feff None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:44.883907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:26:44.890256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9522e-848f-4ba3-aec4-679b87e43638 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:44.890729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9522e-848f-4ba3-aec4-679b87e43638 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:44.891098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9522e-848f-4ba3-aec4-679b87e43638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:44.913545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0bd9522e-848f-4ba3-aec4-679b87e43638 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:26:44.914749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0bd9522e-848f-4ba3-aec4-679b87e43638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:44.923519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:26:45.209107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf4237-910b-4232-959b-3acf4e21feff None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:26:45.215242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=94 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:26:45.222408 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf4237-910b-4232-959b-3acf4e21feff None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:26:45.226252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206bfb0-86cb-4d5d-9f03-b1579c4ed1f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:26:45.226848 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206bfb0-86cb-4d5d-9f03-b1579c4ed1f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:45.227122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206bfb0-86cb-4d5d-9f03-b1579c4ed1f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:45.227389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206bfb0-86cb-4d5d-9f03-b1579c4ed1f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:45.236754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206bfb0-86cb-4d5d-9f03-b1579c4ed1f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:26:45.238233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206bfb0-86cb-4d5d-9f03-b1579c4ed1f8 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:26:45.247826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 764/1533] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:26:48.975863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8bef4-0f8c-4606-9548-189c2e24996e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:48.976436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8bef4-0f8c-4606-9548-189c2e24996e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:48.976436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8bef4-0f8c-4606-9548-189c2e24996e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:48.977804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:26:48.984764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39684b-25b6-45de-a55d-f3cc1cdca9ee None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:48.985511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39684b-25b6-45de-a55d-f3cc1cdca9ee None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:48.985691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39684b-25b6-45de-a55d-f3cc1cdca9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:49.012877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9d39684b-25b6-45de-a55d-f3cc1cdca9ee None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:26:49.014088 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9d39684b-25b6-45de-a55d-f3cc1cdca9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:49.023084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:26:49.323812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8bef4-0f8c-4606-9548-189c2e24996e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:26:49.326867 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7cc8bef4-0f8c-4606-9548-189c2e24996e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:49.425365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8bef4-0f8c-4606-9548-189c2e24996e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:26:49.429481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e85a0-08ee-4444-b0ea-a996bff922a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:26:49.430085 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e85a0-08ee-4444-b0ea-a996bff922a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:49.430361 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e85a0-08ee-4444-b0ea-a996bff922a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:49.430627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e85a0-08ee-4444-b0ea-a996bff922a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:49.440782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e85a0-08ee-4444-b0ea-a996bff922a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:26:49.442267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e85a0-08ee-4444-b0ea-a996bff922a0 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:26:49.452169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 771/1536] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:26:51.605749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074fd4c-ec80-4643-8343-c545a52a5951 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:51.606368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074fd4c-ec80-4643-8343-c545a52a5951 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:51.606368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074fd4c-ec80-4643-8343-c545a52a5951 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:51.608372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 04:26:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:26:51.615930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eda42-5bbe-49fe-8a4d-e0b38458c92b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:51.616406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eda42-5bbe-49fe-8a4d-e0b38458c92b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:51.616502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eda42-5bbe-49fe-8a4d-e0b38458c92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:51.636611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0f3eda42-5bbe-49fe-8a4d-e0b38458c92b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:26:51.637755 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0f3eda42-5bbe-49fe-8a4d-e0b38458c92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:51.646272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 04:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:26:51.948214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074fd4c-ec80-4643-8343-c545a52a5951 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:26:51.950353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8074fd4c-ec80-4643-8343-c545a52a5951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:26:51.966264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074fd4c-ec80-4643-8343-c545a52a5951 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:26:51.971268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e488d08-3b04-4b4f-895d-f2be1bc59cf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:26:51.971865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e488d08-3b04-4b4f-895d-f2be1bc59cf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:26:51.972135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e488d08-3b04-4b4f-895d-f2be1bc59cf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:26:51.972366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e488d08-3b04-4b4f-895d-f2be1bc59cf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:26:51.982597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e488d08-3b04-4b4f-895d-f2be1bc59cf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:26:51.984020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e488d08-3b04-4b4f-895d-f2be1bc59cf8 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:26:51.993667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 767/1539] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:27:01.631703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:27:01.989797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=14 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:15.217936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6273591-6d60-4938-bbe5-20c6a4eea827 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.218638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6273591-6d60-4938-bbe5-20c6a4eea827 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.218638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6273591-6d60-4938-bbe5-20c6a4eea827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.243794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b6273591-6d60-4938-bbe5-20c6a4eea827 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:15.245022 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b6273591-6d60-4938-bbe5-20c6a4eea827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:15.255304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 773/1540] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:28:15.265004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46874c0d-adfd-4c0a-b846-79ef4b43dfc8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.265670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46874c0d-adfd-4c0a-b846-79ef4b43dfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.265670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46874c0d-adfd-4c0a-b846-79ef4b43dfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.294497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-46874c0d-adfd-4c0a-b846-79ef4b43dfc8 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:15.295746 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-46874c0d-adfd-4c0a-b846-79ef4b43dfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:15.315406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 768/1541] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:28:15.320285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6273591-6d60-4938-bbe5-20c6a4eea827 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:15.330586 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6273591-6d60-4938-bbe5-20c6a4eea827 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:15.346007 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9c20c-975c-4fc6-b546-fde1307c7b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:15.346656 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9c20c-975c-4fc6-b546-fde1307c7b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.346938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9c20c-975c-4fc6-b546-fde1307c7b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.347203 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9c20c-975c-4fc6-b546-fde1307c7b0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.347875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9c20c-975c-4fc6-b546-fde1307c7b0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:15.348175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9c20c-975c-4fc6-b546-fde1307c7b0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:15.349263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-cca9c20c-975c-4fc6-b546-fde1307c7b0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:28:15.349263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 17 04:28:15.349263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: Sep 17 04:28:15.350183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9c20c-975c-4fc6-b546-fde1307c7b0f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:15.369118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 774/1542] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:15.377396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46874c0d-adfd-4c0a-b846-79ef4b43dfc8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:15.391444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46874c0d-adfd-4c0a-b846-79ef4b43dfc8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:15.406369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edbf95-d4a1-4aa3-8208-ef43680a0701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:15.407031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edbf95-d4a1-4aa3-8208-ef43680a0701 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.407303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edbf95-d4a1-4aa3-8208-ef43680a0701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.407662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edbf95-d4a1-4aa3-8208-ef43680a0701 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.408483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edbf95-d4a1-4aa3-8208-ef43680a0701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:15.409623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-12edbf95-d4a1-4aa3-8208-ef43680a0701 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:28:15.409623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 17 04:28:15.409623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: Sep 17 04:28:15.411358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edbf95-d4a1-4aa3-8208-ef43680a0701 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:15.442713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d9858-3035-4780-874a-544e0cbb16cd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.443088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d9858-3035-4780-874a-544e0cbb16cd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.443262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d9858-3035-4780-874a-544e0cbb16cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.474736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-323d9858-3035-4780-874a-544e0cbb16cd None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:15.475932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-323d9858-3035-4780-874a-544e0cbb16cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:15.476216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 769/1543] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:15 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:15.481488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edbf95-d4a1-4aa3-8208-ef43680a0701 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:15.572208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 775/1544] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:28:15.591236 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ba06a-9325-45c1-804c-fe2cd0be75ef None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.591591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ba06a-9325-45c1-804c-fe2cd0be75ef None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.591794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ba06a-9325-45c1-804c-fe2cd0be75ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.595437 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edbf95-d4a1-4aa3-8208-ef43680a0701 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:15.611159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f1eaf-4982-4a59-b1a2-b61596d2b1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:15.612161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f1eaf-4982-4a59-b1a2-b61596d2b1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.612415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f1eaf-4982-4a59-b1a2-b61596d2b1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.612688 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f1eaf-4982-4a59-b1a2-b61596d2b1e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.613354 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f1eaf-4982-4a59-b1a2-b61596d2b1e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:15.615288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f1eaf-4982-4a59-b1a2-b61596d2b1e1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:15.619118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1b0ba06a-9325-45c1-804c-fe2cd0be75ef None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:15.620322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1b0ba06a-9325-45c1-804c-fe2cd0be75ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:15.641677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 776/1545] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:28:15.648519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ba06a-9325-45c1-804c-fe2cd0be75ef None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:15.661133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ba06a-9325-45c1-804c-fe2cd0be75ef None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:15.673182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 770/1546] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:15 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:28:15.673566 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52bd9b-0bcd-47b1-94c5-fe80a934c0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:15.674196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52bd9b-0bcd-47b1-94c5-fe80a934c0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.674475 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52bd9b-0bcd-47b1-94c5-fe80a934c0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.674747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52bd9b-0bcd-47b1-94c5-fe80a934c0b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.675499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52bd9b-0bcd-47b1-94c5-fe80a934c0b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:15.675803 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52bd9b-0bcd-47b1-94c5-fe80a934c0b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:15.679684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52bd9b-0bcd-47b1-94c5-fe80a934c0b6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:15.681257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f1eaf-4982-4a59-b1a2-b61596d2b1e1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:15.700180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 777/1547] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:15.710337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52bd9b-0bcd-47b1-94c5-fe80a934c0b6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:15.713038 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f1eaf-4982-4a59-b1a2-b61596d2b1e1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:15.727612 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52bd9b-0bcd-47b1-94c5-fe80a934c0b6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:15.729663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc81de-ddb5-41a1-873b-b70cd6ca78c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:15.730288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc81de-ddb5-41a1-873b-b70cd6ca78c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.730566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc81de-ddb5-41a1-873b-b70cd6ca78c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.730838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc81de-ddb5-41a1-873b-b70cd6ca78c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.737102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc81de-ddb5-41a1-873b-b70cd6ca78c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:15.737102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc81de-ddb5-41a1-873b-b70cd6ca78c2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:15.747860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cd4fd-4c11-4cf0-a90a-389f8c7efe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:15.748834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cd4fd-4c11-4cf0-a90a-389f8c7efe6e admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.749334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 771/1548] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:15.749589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cd4fd-4c11-4cf0-a90a-389f8c7efe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.750079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cd4fd-4c11-4cf0-a90a-389f8c7efe6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.751200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cd4fd-4c11-4cf0-a90a-389f8c7efe6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:15.753450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cd4fd-4c11-4cf0-a90a-389f8c7efe6e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:15.754625 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc81de-ddb5-41a1-873b-b70cd6ca78c2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:15.776683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc81de-ddb5-41a1-873b-b70cd6ca78c2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:15.790535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ed277-9f13-46aa-9926-03aea9219371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:15.791171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ed277-9f13-46aa-9926-03aea9219371 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.791452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ed277-9f13-46aa-9926-03aea9219371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.791724 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ed277-9f13-46aa-9926-03aea9219371 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.792380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ed277-9f13-46aa-9926-03aea9219371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:15.794030 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ed277-9f13-46aa-9926-03aea9219371 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:15.809827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 772/1549] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:15.811243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06009bc1-16e5-4509-b135-7cf1c697bdb4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.811525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06009bc1-16e5-4509-b135-7cf1c697bdb4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.811794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06009bc1-16e5-4509-b135-7cf1c697bdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.828099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 778/1550] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:15 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:15.829939 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71c87a-edab-4484-be7d-e2edfbd14d7e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.830440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71c87a-edab-4484-be7d-e2edfbd14d7e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.830926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71c87a-edab-4484-be7d-e2edfbd14d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.846991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-06009bc1-16e5-4509-b135-7cf1c697bdb4 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:15.848209 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-06009bc1-16e5-4509-b135-7cf1c697bdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:15.863064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7f71c87a-edab-4484-be7d-e2edfbd14d7e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:15.864518 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7f71c87a-edab-4484-be7d-e2edfbd14d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:15.925718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 773/1551] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:15.926555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06009bc1-16e5-4509-b135-7cf1c697bdb4 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:15.935663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06009bc1-16e5-4509-b135-7cf1c697bdb4 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:15.936668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 779/1552] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:15.937835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e389bf-ac2c-4a49-b84a-713e153b132f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.938159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e389bf-ac2c-4a49-b84a-713e153b132f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.938381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e389bf-ac2c-4a49-b84a-713e153b132f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.940773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5792f-81ce-4bb6-bfc3-bd0a59746017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:15.941540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5792f-81ce-4bb6-bfc3-bd0a59746017 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.941760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5792f-81ce-4bb6-bfc3-bd0a59746017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.942034 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5792f-81ce-4bb6-bfc3-bd0a59746017 admin admin] PATH_INFO: `/v3/projects/e9f593f419bb4ebeb351a9ad54a2a532/users/094abcdcd1f54a658564357056e7849a/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.957161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5792f-81ce-4bb6-bfc3-bd0a59746017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f593f419bb4ebeb351a9ad54a2a532, user_id=094abcdcd1f54a658564357056e7849a, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:15.957441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e8e389bf-ac2c-4a49-b84a-713e153b132f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:15.958267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-6bb5792f-81ce-4bb6-bfc3-bd0a59746017 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:28:15.958267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 17 04:28:15.958267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: Sep 17 04:28:15.958637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e8e389bf-ac2c-4a49-b84a-713e153b132f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:15.960192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5792f-81ce-4bb6-bfc3-bd0a59746017 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:15.966599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 780/1553] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:15.967510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e389bf-ac2c-4a49-b84a-713e153b132f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:15.976981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e389bf-ac2c-4a49-b84a-713e153b132f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:15.979457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 774/1554] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:15 2022] PUT /identity/v3/projects/e9f593f419bb4ebeb351a9ad54a2a532/users/094abcdcd1f54a658564357056e7849a/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:15.980658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63649daa-abf4-4e64-8f85-a51ec367b397 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:15.980940 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63649daa-abf4-4e64-8f85-a51ec367b397 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:15.981266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63649daa-abf4-4e64-8f85-a51ec367b397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:15.995454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:16.001885 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-63649daa-abf4-4e64-8f85-a51ec367b397 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:16.003745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-63649daa-abf4-4e64-8f85-a51ec367b397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:16.009229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:16.038302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a519b83-b7af-4345-a3c7-0fddf2077923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.038915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a519b83-b7af-4345-a3c7-0fddf2077923 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.039189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a519b83-b7af-4345-a3c7-0fddf2077923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.039457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a519b83-b7af-4345-a3c7-0fddf2077923 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.040085 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a519b83-b7af-4345-a3c7-0fddf2077923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.041122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-6a519b83-b7af-4345-a3c7-0fddf2077923 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:28:16.041122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 17 04:28:16.041122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: Sep 17 04:28:16.042319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a519b83-b7af-4345-a3c7-0fddf2077923 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.063074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 775/1555] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:16.064063 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 781/1556] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:15 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:16.065029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51247b83-3aef-43de-b540-4596d5e6e6b9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.065459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2d0bb-2805-461f-a71b-519426065bcb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.065746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2d0bb-2805-461f-a71b-519426065bcb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.065925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51247b83-3aef-43de-b540-4596d5e6e6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.066040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2d0bb-2805-461f-a71b-519426065bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.066481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51247b83-3aef-43de-b540-4596d5e6e6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.085976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-51247b83-3aef-43de-b540-4596d5e6e6b9 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:16.086216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-68e2d0bb-2805-461f-a71b-519426065bcb None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:16.087168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-51247b83-3aef-43de-b540-4596d5e6e6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:16.087288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-68e2d0bb-2805-461f-a71b-519426065bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:16.095005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 782/1557] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:16.096378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4dd7b-98dd-4750-ac59-d1c6c17e194c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.096663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4dd7b-98dd-4750-ac59-d1c6c17e194c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.096937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4dd7b-98dd-4750-ac59-d1c6c17e194c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.101381 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 776/1558] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:16.103708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22dc80-2885-4e8c-823e-636005af0574 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.103989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22dc80-2885-4e8c-823e-636005af0574 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.104264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22dc80-2885-4e8c-823e-636005af0574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.116880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a8b4dd7b-98dd-4750-ac59-d1c6c17e194c None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:16.118100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a8b4dd7b-98dd-4750-ac59-d1c6c17e194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:16.123362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0c22dc80-2885-4e8c-823e-636005af0574 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:16.124514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0c22dc80-2885-4e8c-823e-636005af0574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:16.125848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 783/1559] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:16.127208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84acb65-328c-4f88-ab58-a42bfa70b393 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.127486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84acb65-328c-4f88-ab58-a42bfa70b393 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.127761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84acb65-328c-4f88-ab58-a42bfa70b393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.135902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 777/1560] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:16.137360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e46749-80ca-4062-a464-6d9705a20d5f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.137637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e46749-80ca-4062-a464-6d9705a20d5f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.137902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e46749-80ca-4062-a464-6d9705a20d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.150542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c84acb65-328c-4f88-ab58-a42bfa70b393 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:16.151811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c84acb65-328c-4f88-ab58-a42bfa70b393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:16.156467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f9e46749-80ca-4062-a464-6d9705a20d5f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:16.157713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f9e46749-80ca-4062-a464-6d9705a20d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:16.160040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 784/1561] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:16.160831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84acb65-328c-4f88-ab58-a42bfa70b393 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.165391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 778/1562] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:16.166647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1260c-b4e5-4017-9b59-63b08e8f87e5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.166928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1260c-b4e5-4017-9b59-63b08e8f87e5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.167194 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1260c-b4e5-4017-9b59-63b08e8f87e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.173118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84acb65-328c-4f88-ab58-a42bfa70b393 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.177974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed51e66-c98c-4abc-9600-c0eacdd39c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.178578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed51e66-c98c-4abc-9600-c0eacdd39c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.178856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed51e66-c98c-4abc-9600-c0eacdd39c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.179129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed51e66-c98c-4abc-9600-c0eacdd39c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.179773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed51e66-c98c-4abc-9600-c0eacdd39c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.181415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed51e66-c98c-4abc-9600-c0eacdd39c4f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.186310 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-81a1260c-b4e5-4017-9b59-63b08e8f87e5 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:16.187480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-81a1260c-b4e5-4017-9b59-63b08e8f87e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:16.194789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 785/1563] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:16.194789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7086a-c0a4-4555-a86c-c072560cb02f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.194789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7086a-c0a4-4555-a86c-c072560cb02f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.194789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7086a-c0a4-4555-a86c-c072560cb02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.195761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 779/1564] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:16.196768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499945d6-6bc7-41f0-8e64-5f2f166fce27 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.197045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499945d6-6bc7-41f0-8e64-5f2f166fce27 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.197341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499945d6-6bc7-41f0-8e64-5f2f166fce27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.213126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bac7086a-c0a4-4555-a86c-c072560cb02f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:16.214317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bac7086a-c0a4-4555-a86c-c072560cb02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:16.215697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-499945d6-6bc7-41f0-8e64-5f2f166fce27 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:16.216854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-499945d6-6bc7-41f0-8e64-5f2f166fce27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:16.222108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 786/1565] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:16.226266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7086a-c0a4-4555-a86c-c072560cb02f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.227035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 780/1566] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:16.228176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499945d6-6bc7-41f0-8e64-5f2f166fce27 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.236514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7086a-c0a4-4555-a86c-c072560cb02f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.237379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499945d6-6bc7-41f0-8e64-5f2f166fce27 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.241152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422bfcc-9cd3-4eb8-b421-15f28e9bbab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.241741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422bfcc-9cd3-4eb8-b421-15f28e9bbab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.241865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c368b-b163-4d62-bb82-071091f9d2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.242017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422bfcc-9cd3-4eb8-b421-15f28e9bbab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.242286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422bfcc-9cd3-4eb8-b421-15f28e9bbab2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.242434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c368b-b163-4d62-bb82-071091f9d2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.242710 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c368b-b163-4d62-bb82-071091f9d2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.242927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422bfcc-9cd3-4eb8-b421-15f28e9bbab2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:16.243030 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c368b-b163-4d62-bb82-071091f9d2e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.243224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422bfcc-9cd3-4eb8-b421-15f28e9bbab2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.243594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c368b-b163-4d62-bb82-071091f9d2e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:16.243893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c368b-b163-4d62-bb82-071091f9d2e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.244851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422bfcc-9cd3-4eb8-b421-15f28e9bbab2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.245537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c368b-b163-4d62-bb82-071091f9d2e4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.256598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 787/1567] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:16.257215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 781/1568] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:16.257870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c368b-b163-4d62-bb82-071091f9d2e4 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.261594 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422bfcc-9cd3-4eb8-b421-15f28e9bbab2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.272840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422bfcc-9cd3-4eb8-b421-15f28e9bbab2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.277353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f33f78-44de-4f88-98e6-3a891af9d1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.277969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f33f78-44de-4f88-98e6-3a891af9d1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.278241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f33f78-44de-4f88-98e6-3a891af9d1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.278505 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f33f78-44de-4f88-98e6-3a891af9d1a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.279138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f33f78-44de-4f88-98e6-3a891af9d1a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:16.279429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f33f78-44de-4f88-98e6-3a891af9d1a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.281070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f33f78-44de-4f88-98e6-3a891af9d1a2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.300170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 788/1569] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:16.300170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f33f78-44de-4f88-98e6-3a891af9d1a2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.308392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f33f78-44de-4f88-98e6-3a891af9d1a2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.313513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c368b-b163-4d62-bb82-071091f9d2e4 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.314106 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed3b54-eeb9-4d24-8762-e98142218245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.314250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed3b54-eeb9-4d24-8762-e98142218245 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.314402 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed3b54-eeb9-4d24-8762-e98142218245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.314673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed3b54-eeb9-4d24-8762-e98142218245 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.315332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed3b54-eeb9-4d24-8762-e98142218245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.316890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed3b54-eeb9-4d24-8762-e98142218245 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.317220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe314004-a452-4566-9df6-9566dc4927cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.317825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe314004-a452-4566-9df6-9566dc4927cf service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.318099 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe314004-a452-4566-9df6-9566dc4927cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.318366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe314004-a452-4566-9df6-9566dc4927cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.326717 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 789/1570] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:16.327428 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe314004-a452-4566-9df6-9566dc4927cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.327541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed3b54-eeb9-4d24-8762-e98142218245 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.328977 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe314004-a452-4566-9df6-9566dc4927cf service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.336656 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed3b54-eeb9-4d24-8762-e98142218245 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.338889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 782/1571] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:16.339560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe314004-a452-4566-9df6-9566dc4927cf service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.348571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe314004-a452-4566-9df6-9566dc4927cf service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.351725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594a255-9ae6-40c1-806a-50916fb9706e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.352335 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594a255-9ae6-40c1-806a-50916fb9706e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.352609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594a255-9ae6-40c1-806a-50916fb9706e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.353285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594a255-9ae6-40c1-806a-50916fb9706e admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.353931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594a255-9ae6-40c1-806a-50916fb9706e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:16.354258 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594a255-9ae6-40c1-806a-50916fb9706e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.355893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594a255-9ae6-40c1-806a-50916fb9706e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.358066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ba941-d075-4b27-87d3-4e75b45d3923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.359147 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ba941-d075-4b27-87d3-4e75b45d3923 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.359428 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ba941-d075-4b27-87d3-4e75b45d3923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.359698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ba941-d075-4b27-87d3-4e75b45d3923 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.360470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ba941-d075-4b27-87d3-4e75b45d3923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:16.360878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ba941-d075-4b27-87d3-4e75b45d3923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.363077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ba941-d075-4b27-87d3-4e75b45d3923 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.368179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 790/1572] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:16.369006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594a255-9ae6-40c1-806a-50916fb9706e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.378917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594a255-9ae6-40c1-806a-50916fb9706e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.387505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 783/1573] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:16.387505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ba941-d075-4b27-87d3-4e75b45d3923 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.392431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ba941-d075-4b27-87d3-4e75b45d3923 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.393942 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1149af3-164f-44b0-a5bc-e779073a4827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.394561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1149af3-164f-44b0-a5bc-e779073a4827 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.394883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1149af3-164f-44b0-a5bc-e779073a4827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.395159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1149af3-164f-44b0-a5bc-e779073a4827 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.395872 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1149af3-164f-44b0-a5bc-e779073a4827 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:16.396175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1149af3-164f-44b0-a5bc-e779073a4827 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.397276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd88aae-5bbb-4ce2-bc51-0790da220775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.397862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1149af3-164f-44b0-a5bc-e779073a4827 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.397979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd88aae-5bbb-4ce2-bc51-0790da220775 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.398151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd88aae-5bbb-4ce2-bc51-0790da220775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.398571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd88aae-5bbb-4ce2-bc51-0790da220775 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.399243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd88aae-5bbb-4ce2-bc51-0790da220775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.401158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd88aae-5bbb-4ce2-bc51-0790da220775 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.417115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 791/1574] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:16.417115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1149af3-164f-44b0-a5bc-e779073a4827 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.423697 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1149af3-164f-44b0-a5bc-e779073a4827 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.429880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be0592-4382-4416-8e11-5ca80b74d795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.430572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be0592-4382-4416-8e11-5ca80b74d795 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.430790 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be0592-4382-4416-8e11-5ca80b74d795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.431062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be0592-4382-4416-8e11-5ca80b74d795 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.431912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be0592-4382-4416-8e11-5ca80b74d795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.434131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be0592-4382-4416-8e11-5ca80b74d795 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.441635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 784/1575] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:16.445653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd88aae-5bbb-4ce2-bc51-0790da220775 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.468777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 792/1576] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:16.468777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be0592-4382-4416-8e11-5ca80b74d795 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.481455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:16.518086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd88aae-5bbb-4ce2-bc51-0790da220775 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.542839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be0592-4382-4416-8e11-5ca80b74d795 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.547706 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641e230-01f9-4649-8e26-fff86a911818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.548583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641e230-01f9-4649-8e26-fff86a911818 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.548864 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641e230-01f9-4649-8e26-fff86a911818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.549166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641e230-01f9-4649-8e26-fff86a911818 admin admin] PATH_INFO: `/v3/projects/e0b194b28dec41119ae0d6d913f19c31/users/ed4e695113f843fda52ce62792a36bb0/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.551548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:16.567146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641e230-01f9-4649-8e26-fff86a911818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b194b28dec41119ae0d6d913f19c31, user_id=ed4e695113f843fda52ce62792a36bb0, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.573509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641e230-01f9-4649-8e26-fff86a911818 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.598048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb68a8-9952-4b32-87b2-3f3f9b849db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.598682 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb68a8-9952-4b32-87b2-3f3f9b849db1 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.598963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb68a8-9952-4b32-87b2-3f3f9b849db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.599237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb68a8-9952-4b32-87b2-3f3f9b849db1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.599884 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb68a8-9952-4b32-87b2-3f3f9b849db1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.600003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 793/1577] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:16 2022] PUT /identity/v3/projects/e0b194b28dec41119ae0d6d913f19c31/users/ed4e695113f843fda52ce62792a36bb0/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:16.601030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641e230-01f9-4649-8e26-fff86a911818 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.602077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb68a8-9952-4b32-87b2-3f3f9b849db1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.633392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 785/1578] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:16.634577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb68a8-9952-4b32-87b2-3f3f9b849db1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.677483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641e230-01f9-4649-8e26-fff86a911818 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.703701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb68a8-9952-4b32-87b2-3f3f9b849db1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.708461 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b268f2e-13c3-48c7-b88c-2f1b39f623ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.709197 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b268f2e-13c3-48c7-b88c-2f1b39f623ca admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.709596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b268f2e-13c3-48c7-b88c-2f1b39f623ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.710120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b268f2e-13c3-48c7-b88c-2f1b39f623ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.710797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b268f2e-13c3-48c7-b88c-2f1b39f623ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.713248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b268f2e-13c3-48c7-b88c-2f1b39f623ca admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.743141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60d112-bfa5-4c0d-900e-d3b7dcbf4d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.743758 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60d112-bfa5-4c0d-900e-d3b7dcbf4d8c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.744032 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60d112-bfa5-4c0d-900e-d3b7dcbf4d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.744298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60d112-bfa5-4c0d-900e-d3b7dcbf4d8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.747826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 786/1579] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:16.748691 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60d112-bfa5-4c0d-900e-d3b7dcbf4d8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.748825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b268f2e-13c3-48c7-b88c-2f1b39f623ca admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.748926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60d112-bfa5-4c0d-900e-d3b7dcbf4d8c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.785176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 794/1580] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/projects => generated 369 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:16.785176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60d112-bfa5-4c0d-900e-d3b7dcbf4d8c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.802024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:16.817027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b268f2e-13c3-48c7-b88c-2f1b39f623ca admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.872321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60d112-bfa5-4c0d-900e-d3b7dcbf4d8c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.878155 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50871-ea68-4da3-86f1-189e6ebdf93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.879199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50871-ea68-4da3-86f1-189e6ebdf93c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.879671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50871-ea68-4da3-86f1-189e6ebdf93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.880230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50871-ea68-4da3-86f1-189e6ebdf93c admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.880538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cb475-2fb1-4800-b5f8-66713ab024f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.881406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cb475-2fb1-4800-b5f8-66713ab024f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.881580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe50871-ea68-4da3-86f1-189e6ebdf93c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.881740 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cb475-2fb1-4800-b5f8-66713ab024f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.882366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cb475-2fb1-4800-b5f8-66713ab024f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.883185 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cb475-2fb1-4800-b5f8-66713ab024f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:16.884557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe50871-ea68-4da3-86f1-189e6ebdf93c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.885622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cb475-2fb1-4800-b5f8-66713ab024f0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:16.914685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 795/1581] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:16.915924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50871-ea68-4da3-86f1-189e6ebdf93c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:16.925934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 787/1582] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/projects => generated 367 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:16.926751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:16.927242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e898b01-d4be-424f-8b1a-304061b4b745 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.928538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e898b01-d4be-424f-8b1a-304061b4b745 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.928820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e898b01-d4be-424f-8b1a-304061b4b745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.930689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50871-ea68-4da3-86f1-189e6ebdf93c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:16.950482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6e898b01-d4be-424f-8b1a-304061b4b745 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:16.951698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6e898b01-d4be-424f-8b1a-304061b4b745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:16.996365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c4c8f-f055-4e86-a351-e7f8bd49579b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:16.997014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c4c8f-f055-4e86-a351-e7f8bd49579b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:16.997337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c4c8f-f055-4e86-a351-e7f8bd49579b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:16.997644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c4c8f-f055-4e86-a351-e7f8bd49579b admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:16.998287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c4c8f-f055-4e86-a351-e7f8bd49579b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.000290 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c4c8f-f055-4e86-a351-e7f8bd49579b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.016324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 788/1583] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:17.017244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e898b01-d4be-424f-8b1a-304061b4b745 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.027481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 796/1584] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:16 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:17.027481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c4c8f-f055-4e86-a351-e7f8bd49579b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.038128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e898b01-d4be-424f-8b1a-304061b4b745 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.039834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c4c8f-f055-4e86-a351-e7f8bd49579b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.042911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb8da5-8d79-48b5-9408-7996bc7bdb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.043535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb8da5-8d79-48b5-9408-7996bc7bdb39 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.043827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb8da5-8d79-48b5-9408-7996bc7bdb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.044113 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb8da5-8d79-48b5-9408-7996bc7bdb39 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.044724 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb8da5-8d79-48b5-9408-7996bc7bdb39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.045281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae2cad-efa7-4b0d-af7b-4f51c0b4b628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.045879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae2cad-efa7-4b0d-af7b-4f51c0b4b628 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.046356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae2cad-efa7-4b0d-af7b-4f51c0b4b628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.046629 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae2cad-efa7-4b0d-af7b-4f51c0b4b628 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.046833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb8da5-8d79-48b5-9408-7996bc7bdb39 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.047252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae2cad-efa7-4b0d-af7b-4f51c0b4b628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.049453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae2cad-efa7-4b0d-af7b-4f51c0b4b628 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.070751 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 789/1585] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:17 2022] POST /identity/v3/users => generated 291 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:17.071368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb8da5-8d79-48b5-9408-7996bc7bdb39 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.077831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 797/1586] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:17 2022] POST /identity/v3/users => generated 290 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:17.078582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae2cad-efa7-4b0d-af7b-4f51c0b4b628 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.084415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb8da5-8d79-48b5-9408-7996bc7bdb39 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.089886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4526bb-d7ac-4155-8b78-21ed398aef79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.090492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4526bb-d7ac-4155-8b78-21ed398aef79 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.090770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4526bb-d7ac-4155-8b78-21ed398aef79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.090884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae2cad-efa7-4b0d-af7b-4f51c0b4b628 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.091093 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4526bb-d7ac-4155-8b78-21ed398aef79 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.091667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4526bb-d7ac-4155-8b78-21ed398aef79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.093648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4526bb-d7ac-4155-8b78-21ed398aef79 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.095560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03916f2-6a47-41bd-9674-4c7484fc8f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.096256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03916f2-6a47-41bd-9674-4c7484fc8f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.096530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03916f2-6a47-41bd-9674-4c7484fc8f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.096816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03916f2-6a47-41bd-9674-4c7484fc8f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.097699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03916f2-6a47-41bd-9674-4c7484fc8f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.099302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03916f2-6a47-41bd-9674-4c7484fc8f67 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.117238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 798/1587] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:17.117238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03916f2-6a47-41bd-9674-4c7484fc8f67 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.126557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 790/1588] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:17 2022] POST /identity/v3/users => generated 285 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:17.127512 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4526bb-d7ac-4155-8b78-21ed398aef79 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.128344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03916f2-6a47-41bd-9674-4c7484fc8f67 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.134956 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008457b1-85ff-4796-a45b-2d7e3ab68b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.135713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008457b1-85ff-4796-a45b-2d7e3ab68b85 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.136077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008457b1-85ff-4796-a45b-2d7e3ab68b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.136456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008457b1-85ff-4796-a45b-2d7e3ab68b85 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.137196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008457b1-85ff-4796-a45b-2d7e3ab68b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.140342 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008457b1-85ff-4796-a45b-2d7e3ab68b85 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.177887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 799/1589] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:17 2022] POST /identity/v3/users => generated 284 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:17.180479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008457b1-85ff-4796-a45b-2d7e3ab68b85 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.192271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008457b1-85ff-4796-a45b-2d7e3ab68b85 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.195892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:17.198226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7450c05-f177-46eb-9dc9-31b136456476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.199538 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7450c05-f177-46eb-9dc9-31b136456476 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.199976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7450c05-f177-46eb-9dc9-31b136456476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.200391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7450c05-f177-46eb-9dc9-31b136456476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.201315 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7450c05-f177-46eb-9dc9-31b136456476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.203225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7450c05-f177-46eb-9dc9-31b136456476 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.207608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4526bb-d7ac-4155-8b78-21ed398aef79 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.213056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b957-ff11-4a07-8cc0-740f9aa1b4fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.213841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b957-ff11-4a07-8cc0-740f9aa1b4fc service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.214096 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b957-ff11-4a07-8cc0-740f9aa1b4fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.214376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b957-ff11-4a07-8cc0-740f9aa1b4fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.221353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 800/1590] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:17.222139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7450c05-f177-46eb-9dc9-31b136456476 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.225848 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b957-ff11-4a07-8cc0-740f9aa1b4fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.227370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b957-ff11-4a07-8cc0-740f9aa1b4fc service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.236492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7450c05-f177-46eb-9dc9-31b136456476 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.241822 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 791/1591] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:17.243620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106e379-a6e4-424a-802e-11a48d94d3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.244360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106e379-a6e4-424a-802e-11a48d94d3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.244659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b957-ff11-4a07-8cc0-740f9aa1b4fc service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.244845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106e379-a6e4-424a-802e-11a48d94d3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.245076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106e379-a6e4-424a-802e-11a48d94d3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.246161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106e379-a6e4-424a-802e-11a48d94d3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.250169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106e379-a6e4-424a-802e-11a48d94d3d9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.256772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b957-ff11-4a07-8cc0-740f9aa1b4fc service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.263821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742de0c1-239a-4a41-a035-fd2aad2f4fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.265736 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742de0c1-239a-4a41-a035-fd2aad2f4fbb admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.266087 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742de0c1-239a-4a41-a035-fd2aad2f4fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.266298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 801/1592] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:17.266430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742de0c1-239a-4a41-a035-fd2aad2f4fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.267518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742de0c1-239a-4a41-a035-fd2aad2f4fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.268154 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106e379-a6e4-424a-802e-11a48d94d3d9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.269334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742de0c1-239a-4a41-a035-fd2aad2f4fbb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.280032 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106e379-a6e4-424a-802e-11a48d94d3d9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.282102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 792/1593] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:17.283236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742de0c1-239a-4a41-a035-fd2aad2f4fbb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.291413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950147c-5e3c-4bf4-95c2-4dd3d251d1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.292186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950147c-5e3c-4bf4-95c2-4dd3d251d1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.292472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950147c-5e3c-4bf4-95c2-4dd3d251d1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.292733 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950147c-5e3c-4bf4-95c2-4dd3d251d1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.293415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b950147c-5e3c-4bf4-95c2-4dd3d251d1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.294632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742de0c1-239a-4a41-a035-fd2aad2f4fbb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.295494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b950147c-5e3c-4bf4-95c2-4dd3d251d1a6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.300890 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ca36c-2ae8-4741-955c-a96828c32c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.301477 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ca36c-2ae8-4741-955c-a96828c32c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.302255 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ca36c-2ae8-4741-955c-a96828c32c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.302535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ca36c-2ae8-4741-955c-a96828c32c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.303322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ca36c-2ae8-4741-955c-a96828c32c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.305528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ca36c-2ae8-4741-955c-a96828c32c1c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.307533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 802/1594] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:17.308444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950147c-5e3c-4bf4-95c2-4dd3d251d1a6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.319731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950147c-5e3c-4bf4-95c2-4dd3d251d1a6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.321111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 793/1595] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:17.322085 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ca36c-2ae8-4741-955c-a96828c32c1c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.327182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66c3dd-2aaf-4579-b82b-d26d0b7eb807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.327962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66c3dd-2aaf-4579-b82b-d26d0b7eb807 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.328231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66c3dd-2aaf-4579-b82b-d26d0b7eb807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.328495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66c3dd-2aaf-4579-b82b-d26d0b7eb807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.329490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66c3dd-2aaf-4579-b82b-d26d0b7eb807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.331326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66c3dd-2aaf-4579-b82b-d26d0b7eb807 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.333155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ca36c-2ae8-4741-955c-a96828c32c1c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.340234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88866431-7c00-4321-90fb-019490fafc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.340974 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88866431-7c00-4321-90fb-019490fafc48 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.341340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88866431-7c00-4321-90fb-019490fafc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.341614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88866431-7c00-4321-90fb-019490fafc48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.342262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88866431-7c00-4321-90fb-019490fafc48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.346773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88866431-7c00-4321-90fb-019490fafc48 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.346953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 803/1596] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:17.346953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66c3dd-2aaf-4579-b82b-d26d0b7eb807 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.361129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66c3dd-2aaf-4579-b82b-d26d0b7eb807 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.361340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 794/1597] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:17.361340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88866431-7c00-4321-90fb-019490fafc48 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.364759 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8b297-2a03-45ee-8c53-70065600a424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.365407 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8b297-2a03-45ee-8c53-70065600a424 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.365681 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8b297-2a03-45ee-8c53-70065600a424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.366008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8b297-2a03-45ee-8c53-70065600a424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.366647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8b297-2a03-45ee-8c53-70065600a424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.368272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8b297-2a03-45ee-8c53-70065600a424 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.376349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88866431-7c00-4321-90fb-019490fafc48 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.381358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 804/1598] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:17.382540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e09a10-c467-4cfa-a3db-6f79b3f33a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.382699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8b297-2a03-45ee-8c53-70065600a424 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.383446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e09a10-c467-4cfa-a3db-6f79b3f33a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.383737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e09a10-c467-4cfa-a3db-6f79b3f33a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.384015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e09a10-c467-4cfa-a3db-6f79b3f33a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.385034 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e09a10-c467-4cfa-a3db-6f79b3f33a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.387292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e09a10-c467-4cfa-a3db-6f79b3f33a34 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.399332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8b297-2a03-45ee-8c53-70065600a424 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.405102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 795/1599] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:17.405967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e09a10-c467-4cfa-a3db-6f79b3f33a34 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.408260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559e078-d65a-4238-a029-23949626a073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.410199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559e078-d65a-4238-a029-23949626a073 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.410636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559e078-d65a-4238-a029-23949626a073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.411251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559e078-d65a-4238-a029-23949626a073 admin admin] PATH_INFO: `/v3/projects/39cc778e84014ed98f33554b7c46b174/users/9be671f062ed4cb793facf9ca7920263/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.424909 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e09a10-c467-4cfa-a3db-6f79b3f33a34 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.430393 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee068b38-aeef-44ce-a6c3-c21c8040ddce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.430783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee068b38-aeef-44ce-a6c3-c21c8040ddce admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.431170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee068b38-aeef-44ce-a6c3-c21c8040ddce admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.431846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee068b38-aeef-44ce-a6c3-c21c8040ddce admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.432329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee068b38-aeef-44ce-a6c3-c21c8040ddce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.434156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee068b38-aeef-44ce-a6c3-c21c8040ddce admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.440887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559e078-d65a-4238-a029-23949626a073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39cc778e84014ed98f33554b7c46b174, user_id=9be671f062ed4cb793facf9ca7920263, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.443776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559e078-d65a-4238-a029-23949626a073 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.448374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 796/1600] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:17.449676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee068b38-aeef-44ce-a6c3-c21c8040ddce admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.464517 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 805/1601] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:17 2022] PUT /identity/v3/projects/39cc778e84014ed98f33554b7c46b174/users/9be671f062ed4cb793facf9ca7920263/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:17.464990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee068b38-aeef-44ce-a6c3-c21c8040ddce admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.470456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559e078-d65a-4238-a029-23949626a073 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.506535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:17.549022 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559e078-d65a-4238-a029-23949626a073 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.549980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c2d8e-0c94-4954-98ad-25edd21dec5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.550705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c2d8e-0c94-4954-98ad-25edd21dec5b admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.550984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c2d8e-0c94-4954-98ad-25edd21dec5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.551299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c2d8e-0c94-4954-98ad-25edd21dec5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.552506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155c2d8e-0c94-4954-98ad-25edd21dec5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.554187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155c2d8e-0c94-4954-98ad-25edd21dec5b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.555035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c8544-40ca-47df-99da-5d9397ddaf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.556119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c8544-40ca-47df-99da-5d9397ddaf56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.556412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c8544-40ca-47df-99da-5d9397ddaf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.557117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c8544-40ca-47df-99da-5d9397ddaf56 admin admin] PATH_INFO: `/v3/projects/8789a927bb6442da93e546b74955f415/users/8a4732d60b5441d8bd2f0896d64ca832/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.572181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 797/1602] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:17.572967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c2d8e-0c94-4954-98ad-25edd21dec5b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.578827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c8544-40ca-47df-99da-5d9397ddaf56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8789a927bb6442da93e546b74955f415, user_id=8a4732d60b5441d8bd2f0896d64ca832, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.581210 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c8544-40ca-47df-99da-5d9397ddaf56 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.584703 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c2d8e-0c94-4954-98ad-25edd21dec5b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.591136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040f726-c283-4ad7-97e4-cd3e0cf2bec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.593071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040f726-c283-4ad7-97e4-cd3e0cf2bec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.593459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040f726-c283-4ad7-97e4-cd3e0cf2bec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.593726 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040f726-c283-4ad7-97e4-cd3e0cf2bec8 admin admin] PATH_INFO: `/v3/projects/5af63acf1c584cb4a4a55e9e53f05068/users/6eb1a37b6cc0412b987ee04603cba5e1/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.603104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 806/1603] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:17 2022] PUT /identity/v3/projects/8789a927bb6442da93e546b74955f415/users/8a4732d60b5441d8bd2f0896d64ca832/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:17.605896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c8544-40ca-47df-99da-5d9397ddaf56 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.617907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040f726-c283-4ad7-97e4-cd3e0cf2bec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5af63acf1c584cb4a4a55e9e53f05068, user_id=6eb1a37b6cc0412b987ee04603cba5e1, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.621315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040f726-c283-4ad7-97e4-cd3e0cf2bec8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.645666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 798/1604] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:17 2022] PUT /identity/v3/projects/5af63acf1c584cb4a4a55e9e53f05068/users/6eb1a37b6cc0412b987ee04603cba5e1/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:17.646725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040f726-c283-4ad7-97e4-cd3e0cf2bec8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.706418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c8544-40ca-47df-99da-5d9397ddaf56 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.748437 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040f726-c283-4ad7-97e4-cd3e0cf2bec8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.756213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29708f3d-404a-45b6-b585-10c63dbd84d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.757047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29708f3d-404a-45b6-b585-10c63dbd84d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.757448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29708f3d-404a-45b6-b585-10c63dbd84d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.757801 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29708f3d-404a-45b6-b585-10c63dbd84d1 admin admin] PATH_INFO: `/v3/projects/901517d15c4b4c3bae31c3389ff50301/users/4742f609c089410c952235906ad98bcd/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.776793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:17.776886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27d434-f8c1-47cf-8a12-926a8f432eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:17.776886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27d434-f8c1-47cf-8a12-926a8f432eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.776886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27d434-f8c1-47cf-8a12-926a8f432eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.777672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27d434-f8c1-47cf-8a12-926a8f432eb5 admin admin] PATH_INFO: `/v3/projects/6292f5855c2a437daa4b82fe05ac7eb8/users/c520e361f89545cea9b84b0a349a3462/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.798219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27d434-f8c1-47cf-8a12-926a8f432eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6292f5855c2a437daa4b82fe05ac7eb8, user_id=c520e361f89545cea9b84b0a349a3462, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.798801 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29708f3d-404a-45b6-b585-10c63dbd84d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901517d15c4b4c3bae31c3389ff50301, user_id=4742f609c089410c952235906ad98bcd, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:17.801510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29708f3d-404a-45b6-b585-10c63dbd84d1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.802526 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27d434-f8c1-47cf-8a12-926a8f432eb5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:17.827101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 807/1605] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:17 2022] PUT /identity/v3/projects/6292f5855c2a437daa4b82fe05ac7eb8/users/c520e361f89545cea9b84b0a349a3462/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:17.827712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 799/1606] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:17 2022] PUT /identity/v3/projects/901517d15c4b4c3bae31c3389ff50301/users/4742f609c089410c952235906ad98bcd/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:17.828762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29708f3d-404a-45b6-b585-10c63dbd84d1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:17.829027 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185788fb-2a54-4475-8cca-b403c01b88ce None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.829473 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185788fb-2a54-4475-8cca-b403c01b88ce None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.830067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185788fb-2a54-4475-8cca-b403c01b88ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:17.881235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-185788fb-2a54-4475-8cca-b403c01b88ce None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:17.882440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-185788fb-2a54-4475-8cca-b403c01b88ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:17.910989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29708f3d-404a-45b6-b585-10c63dbd84d1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:17.939490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:17.983546 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 808/1607] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:17.985995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c841e7f-1c0d-456f-a30e-4b235799bb3a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:17.986292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c841e7f-1c0d-456f-a30e-4b235799bb3a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:17.986569 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c841e7f-1c0d-456f-a30e-4b235799bb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:18.000496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d4d3a-ad7f-4d26-b6b6-358e9675fd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:18.001567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d4d3a-ad7f-4d26-b6b6-358e9675fd77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:18.002177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d4d3a-ad7f-4d26-b6b6-358e9675fd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:18.002606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d4d3a-ad7f-4d26-b6b6-358e9675fd77 admin admin] PATH_INFO: `/v3/projects/f9a4026cd378405d9320d9d4dae775ac/users/3546689f8f93481c89322ff9fe4dbcec/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:18.029200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d4d3a-ad7f-4d26-b6b6-358e9675fd77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a4026cd378405d9320d9d4dae775ac, user_id=3546689f8f93481c89322ff9fe4dbcec, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:18.032055 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3c841e7f-1c0d-456f-a30e-4b235799bb3a None None] MFA Rules not processed for user `8a4732d60b5441d8bd2f0896d64ca832`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:18.033702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3c841e7f-1c0d-456f-a30e-4b235799bb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:18.033821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d4d3a-ad7f-4d26-b6b6-358e9675fd77 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:18.065436 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 800/1608] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:17 2022] PUT /identity/v3/projects/f9a4026cd378405d9320d9d4dae775ac/users/3546689f8f93481c89322ff9fe4dbcec/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:18.075479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0e252-65cd-4c2d-bc3f-fcdd7786dee9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:18.075925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0e252-65cd-4c2d-bc3f-fcdd7786dee9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:18.076211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0e252-65cd-4c2d-bc3f-fcdd7786dee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:18.150076 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-72f0e252-65cd-4c2d-bc3f-fcdd7786dee9 None None] MFA Rules not processed for user `6eb1a37b6cc0412b987ee04603cba5e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:18.151793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-72f0e252-65cd-4c2d-bc3f-fcdd7786dee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:18.199897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 809/1609] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:18.201919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1985f-2b1f-4109-9336-af647a399d1d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:18.202499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1985f-2b1f-4109-9336-af647a399d1d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:18.209382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1985f-2b1f-4109-9336-af647a399d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:18.259997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e3b1985f-2b1f-4109-9336-af647a399d1d None None] MFA Rules not processed for user `c520e361f89545cea9b84b0a349a3462`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:18.263227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e3b1985f-2b1f-4109-9336-af647a399d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:18.349250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 801/1610] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:18.394414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cabab7-3a6b-4243-a124-24cfebc32b3d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:18.394414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cabab7-3a6b-4243-a124-24cfebc32b3d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:18.394414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cabab7-3a6b-4243-a124-24cfebc32b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:18.394414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:18.420700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 810/1611] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:18.423961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1985f-2b1f-4109-9336-af647a399d1d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:18.426480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e3b1985f-2b1f-4109-9336-af647a399d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:18.432935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-01cabab7-3a6b-4243-a124-24cfebc32b3d None None] MFA Rules not processed for user `4742f609c089410c952235906ad98bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:18.434435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-01cabab7-3a6b-4243-a124-24cfebc32b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:18.528911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1985f-2b1f-4109-9336-af647a399d1d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:18.533524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c4b17-dafc-4361-870b-96deedf1f782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:18.534155 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c4b17-dafc-4361-870b-96deedf1f782 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:18.534432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c4b17-dafc-4361-870b-96deedf1f782 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:18.534713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c4b17-dafc-4361-870b-96deedf1f782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:18.561592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:18.587621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 802/1612] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:18.590095 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d1d55-c1d4-46f3-abdf-0c0711ab65dc None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:18.590704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d1d55-c1d4-46f3-abdf-0c0711ab65dc None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:18.594370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d1d55-c1d4-46f3-abdf-0c0711ab65dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:18.637654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-020d1d55-c1d4-46f3-abdf-0c0711ab65dc None None] MFA Rules not processed for user `3546689f8f93481c89322ff9fe4dbcec`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:18.639428 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-020d1d55-c1d4-46f3-abdf-0c0711ab65dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:18.662282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c4b17-dafc-4361-870b-96deedf1f782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:18.664191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c4b17-dafc-4361-870b-96deedf1f782 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:18.684651 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 811/1613] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:18.686265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c579d-7def-4fed-a080-b8ec3542f9a1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:18.686942 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c579d-7def-4fed-a080-b8ec3542f9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:18.686942 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c579d-7def-4fed-a080-b8ec3542f9a1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:18.693468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 812/1614] 10.222.0.60 () {54 vars in 965 bytes} [Sat Sep 17 04:28:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:28:18.695499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c579d-7def-4fed-a080-b8ec3542f9a1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:18.708173 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c579d-7def-4fed-a080-b8ec3542f9a1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:18.716635 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811d778-4782-4a6a-a60c-00278f31a3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:18.717567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811d778-4782-4a6a-a60c-00278f31a3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:18.717950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811d778-4782-4a6a-a60c-00278f31a3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:18.718390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811d778-4782-4a6a-a60c-00278f31a3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:18.719139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811d778-4782-4a6a-a60c-00278f31a3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:18.721649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811d778-4782-4a6a-a60c-00278f31a3f9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:18.781840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 813/1615] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:18 2022] POST /identity/v3/projects => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:18.781956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 803/1616] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:18.781956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d1d55-c1d4-46f3-abdf-0c0711ab65dc None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:18.782608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811d778-4782-4a6a-a60c-00278f31a3f9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:18.803862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d1d55-c1d4-46f3-abdf-0c0711ab65dc None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:18.892263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811d778-4782-4a6a-a60c-00278f31a3f9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:18.902935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb5934-5fea-4aea-87bb-9db5f2e2a231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:18.903473 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb5934-5fea-4aea-87bb-9db5f2e2a231 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:18.903847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb5934-5fea-4aea-87bb-9db5f2e2a231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:18.904393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb5934-5fea-4aea-87bb-9db5f2e2a231 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:18.907064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb5934-5fea-4aea-87bb-9db5f2e2a231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:18.912042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb5934-5fea-4aea-87bb-9db5f2e2a231 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:18.924965 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927df388-5616-4777-a705-2db011aaa266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:18.926314 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927df388-5616-4777-a705-2db011aaa266 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:18.928011 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927df388-5616-4777-a705-2db011aaa266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:18.928390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927df388-5616-4777-a705-2db011aaa266 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:18.930143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927df388-5616-4777-a705-2db011aaa266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:18.935653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927df388-5616-4777-a705-2db011aaa266 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:18.980212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 814/1617] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:18 2022] POST /identity/v3/projects => generated 381 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:18.980212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bc658-c958-42cd-b80b-345c73f78a7c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:18.980212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bc658-c958-42cd-b80b-345c73f78a7c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:18.980212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bc658-c958-42cd-b80b-345c73f78a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:19.002062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 804/1618] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:18 2022] POST /identity/v3/projects => generated 367 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:19.005371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927df388-5616-4777-a705-2db011aaa266 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:19.047654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:19.059813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-225bc658-c958-42cd-b80b-345c73f78a7c None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:19.062069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-225bc658-c958-42cd-b80b-345c73f78a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:19.148798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927df388-5616-4777-a705-2db011aaa266 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:19.166525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e381ab-1e12-4789-b15b-d34917dae152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:19.168028 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e381ab-1e12-4789-b15b-d34917dae152 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:19.168845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e381ab-1e12-4789-b15b-d34917dae152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:19.169320 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e381ab-1e12-4789-b15b-d34917dae152 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:19.173421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e381ab-1e12-4789-b15b-d34917dae152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:19.179506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e381ab-1e12-4789-b15b-d34917dae152 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:19.208150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:19.226846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 805/1619] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:19 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:19.226846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e381ab-1e12-4789-b15b-d34917dae152 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:19.236852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 815/1620] 10.222.0.60 () {58 vars in 1062 bytes} [Sat Sep 17 04:28:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:19.238593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bc658-c958-42cd-b80b-345c73f78a7c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:19.254817 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bc658-c958-42cd-b80b-345c73f78a7c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:19.262488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b118e-833e-49ab-914c-380d9c893cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:19.263826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b118e-833e-49ab-914c-380d9c893cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:19.264343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b118e-833e-49ab-914c-380d9c893cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:19.265037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b118e-833e-49ab-914c-380d9c893cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:19.265875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b118e-833e-49ab-914c-380d9c893cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:19.268658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b118e-833e-49ab-914c-380d9c893cb0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:19.309435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 816/1621] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:19 2022] POST /identity/v3/users => generated 297 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:19.310461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b118e-833e-49ab-914c-380d9c893cb0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:19.336498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b118e-833e-49ab-914c-380d9c893cb0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:19.347265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b3911-c58c-47e1-ab0d-4e91a25e8d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:19.347904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b3911-c58c-47e1-ab0d-4e91a25e8d21 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:19.348184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b3911-c58c-47e1-ab0d-4e91a25e8d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:19.348458 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b3911-c58c-47e1-ab0d-4e91a25e8d21 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:19.349180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b3911-c58c-47e1-ab0d-4e91a25e8d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:19.356863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b3911-c58c-47e1-ab0d-4e91a25e8d21 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:19.361898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e381ab-1e12-4789-b15b-d34917dae152 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:19.368425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981f4d4-3c6b-43e9-ac83-9dd32e73c393 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:19.369676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981f4d4-3c6b-43e9-ac83-9dd32e73c393 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:19.371632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981f4d4-3c6b-43e9-ac83-9dd32e73c393 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:19.372138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981f4d4-3c6b-43e9-ac83-9dd32e73c393 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:19.404171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 817/1622] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:19 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:19.409144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b3911-c58c-47e1-ab0d-4e91a25e8d21 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:19.449608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b3911-c58c-47e1-ab0d-4e91a25e8d21 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:19.452931 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981f4d4-3c6b-43e9-ac83-9dd32e73c393 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:19.454373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981f4d4-3c6b-43e9-ac83-9dd32e73c393 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:19.469242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec20cf-fc93-4eef-bcc1-d482d75ab239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:19.473157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec20cf-fc93-4eef-bcc1-d482d75ab239 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:19.476603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec20cf-fc93-4eef-bcc1-d482d75ab239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:19.476603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec20cf-fc93-4eef-bcc1-d482d75ab239 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:19.476603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec20cf-fc93-4eef-bcc1-d482d75ab239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:19.478173 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec20cf-fc93-4eef-bcc1-d482d75ab239 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:19.478782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 806/1623] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:19.492149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981f4d4-3c6b-43e9-ac83-9dd32e73c393 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:19.523896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981f4d4-3c6b-43e9-ac83-9dd32e73c393 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:19.536852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 818/1624] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:19.538381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec20cf-fc93-4eef-bcc1-d482d75ab239 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:19.539533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36c626-65b8-4c2e-8f50-5ad9fb82b639 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:19.540168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36c626-65b8-4c2e-8f50-5ad9fb82b639 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:19.540453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36c626-65b8-4c2e-8f50-5ad9fb82b639 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:19.540725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36c626-65b8-4c2e-8f50-5ad9fb82b639 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:19.569964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec20cf-fc93-4eef-bcc1-d482d75ab239 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:19.591447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845e917-b64a-47a5-a6cd-313bc9053fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:19.593168 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845e917-b64a-47a5-a6cd-313bc9053fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:19.593697 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845e917-b64a-47a5-a6cd-313bc9053fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:19.594982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845e917-b64a-47a5-a6cd-313bc9053fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:19.595663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845e917-b64a-47a5-a6cd-313bc9053fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:19.606256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845e917-b64a-47a5-a6cd-313bc9053fd7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:19.645461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 819/1625] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:19.646365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845e917-b64a-47a5-a6cd-313bc9053fd7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:19.673469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845e917-b64a-47a5-a6cd-313bc9053fd7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:19.688164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7676ca4-21c8-452d-9585-748441705ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:19.691257 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7676ca4-21c8-452d-9585-748441705ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:19.691257 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7676ca4-21c8-452d-9585-748441705ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:19.691487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7676ca4-21c8-452d-9585-748441705ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:19.691973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7676ca4-21c8-452d-9585-748441705ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:19.693624 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7676ca4-21c8-452d-9585-748441705ba9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:19.711055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:19.716471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36c626-65b8-4c2e-8f50-5ad9fb82b639 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:19.718541 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36c626-65b8-4c2e-8f50-5ad9fb82b639 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:19.731280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 807/1626] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:19.732371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36c626-65b8-4c2e-8f50-5ad9fb82b639 service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:19.744409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 820/1627] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:19.745735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36c626-65b8-4c2e-8f50-5ad9fb82b639 service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:19.749268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7676ca4-21c8-452d-9585-748441705ba9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:19.750801 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942564a-36f0-40be-8815-f04686e8d8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:19.751685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942564a-36f0-40be-8815-f04686e8d8d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:19.752018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942564a-36f0-40be-8815-f04686e8d8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:19.752352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942564a-36f0-40be-8815-f04686e8d8d7 admin admin] PATH_INFO: `/v3/projects/ef1382625553428eb1f818c507cafa7e/users/a37f484345f34966b28d5501cb9b86a6/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:19.770066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942564a-36f0-40be-8815-f04686e8d8d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1382625553428eb1f818c507cafa7e, user_id=a37f484345f34966b28d5501cb9b86a6, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:19.772782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942564a-36f0-40be-8815-f04686e8d8d7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:19.776811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7676ca4-21c8-452d-9585-748441705ba9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:19.787944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c66e7a-b7fe-4f2e-840f-25208d33d1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:19.788970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c66e7a-b7fe-4f2e-840f-25208d33d1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:19.789449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c66e7a-b7fe-4f2e-840f-25208d33d1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:19.789938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c66e7a-b7fe-4f2e-840f-25208d33d1f5 admin admin] PATH_INFO: `/v3/projects/f2183079411d478587a26c465289e3d7/users/d96944e867914a7e859170cf6f30b61f/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:19.792004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 808/1628] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:19 2022] PUT /identity/v3/projects/ef1382625553428eb1f818c507cafa7e/users/a37f484345f34966b28d5501cb9b86a6/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:19.793031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942564a-36f0-40be-8815-f04686e8d8d7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:19.810436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c66e7a-b7fe-4f2e-840f-25208d33d1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2183079411d478587a26c465289e3d7, user_id=d96944e867914a7e859170cf6f30b61f, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:19.813816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c66e7a-b7fe-4f2e-840f-25208d33d1f5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:19.836937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 821/1629] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:19 2022] PUT /identity/v3/projects/f2183079411d478587a26c465289e3d7/users/d96944e867914a7e859170cf6f30b61f/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:19.838979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c66e7a-b7fe-4f2e-840f-25208d33d1f5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:19.876303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942564a-36f0-40be-8815-f04686e8d8d7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:19.960635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826f50c-78ec-4a74-ab13-3398ba39f414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:19.961638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826f50c-78ec-4a74-ab13-3398ba39f414 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:19.962033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826f50c-78ec-4a74-ab13-3398ba39f414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:19.962506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826f50c-78ec-4a74-ab13-3398ba39f414 admin admin] PATH_INFO: `/v3/projects/edad36b182f9411084b55f288aa6afa1/users/1ed919ff26a64b47a6f715c0efad85c7/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:19.983351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826f50c-78ec-4a74-ab13-3398ba39f414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edad36b182f9411084b55f288aa6afa1, user_id=1ed919ff26a64b47a6f715c0efad85c7, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:19.987467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826f50c-78ec-4a74-ab13-3398ba39f414 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:20.005502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c66e7a-b7fe-4f2e-840f-25208d33d1f5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:20.009581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 809/1630] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:19 2022] PUT /identity/v3/projects/edad36b182f9411084b55f288aa6afa1/users/1ed919ff26a64b47a6f715c0efad85c7/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:20.010545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826f50c-78ec-4a74-ab13-3398ba39f414 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:20.121377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826f50c-78ec-4a74-ab13-3398ba39f414 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:20.127750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13574e1-ffe8-4a82-8d11-803c71352d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:20.128582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13574e1-ffe8-4a82-8d11-803c71352d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:20.128866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13574e1-ffe8-4a82-8d11-803c71352d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:20.129184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13574e1-ffe8-4a82-8d11-803c71352d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:20.130144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13574e1-ffe8-4a82-8d11-803c71352d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:20.131940 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13574e1-ffe8-4a82-8d11-803c71352d6a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:20.134488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:20.140782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e57fd8-de79-4e6c-95f3-2bd2aa03fc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:20.141588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e57fd8-de79-4e6c-95f3-2bd2aa03fc45 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:20.142012 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e57fd8-de79-4e6c-95f3-2bd2aa03fc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:20.142379 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e57fd8-de79-4e6c-95f3-2bd2aa03fc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:20.143160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e57fd8-de79-4e6c-95f3-2bd2aa03fc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:20.145359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e57fd8-de79-4e6c-95f3-2bd2aa03fc45 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:20.153898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 810/1631] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:20.153898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13574e1-ffe8-4a82-8d11-803c71352d6a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:20.167733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13574e1-ffe8-4a82-8d11-803c71352d6a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:20.171696 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 822/1632] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:20.172655 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e57fd8-de79-4e6c-95f3-2bd2aa03fc45 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:20.173294 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a3eb4-75ac-4222-942e-9ad0604646a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:20.173906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a3eb4-75ac-4222-942e-9ad0604646a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:20.174181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a3eb4-75ac-4222-942e-9ad0604646a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:20.174448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a3eb4-75ac-4222-942e-9ad0604646a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:20.175420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a3eb4-75ac-4222-942e-9ad0604646a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:20.177035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a3eb4-75ac-4222-942e-9ad0604646a4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:20.186898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:20.196163 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e57fd8-de79-4e6c-95f3-2bd2aa03fc45 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:20.208233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14de50f-4f61-484d-8af8-89109cdb7f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:20.209337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14de50f-4f61-484d-8af8-89109cdb7f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:20.209775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14de50f-4f61-484d-8af8-89109cdb7f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:20.210161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14de50f-4f61-484d-8af8-89109cdb7f97 admin admin] PATH_INFO: `/v3/projects/f2183079411d478587a26c465289e3d7/users/d96944e867914a7e859170cf6f30b61f/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:20.213924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14de50f-4f61-484d-8af8-89109cdb7f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2183079411d478587a26c465289e3d7, user_id=d96944e867914a7e859170cf6f30b61f, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:20.215601 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 811/1633] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:20.216788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a3eb4-75ac-4222-942e-9ad0604646a4 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:20.221352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14de50f-4f61-484d-8af8-89109cdb7f97 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:20.235953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a3eb4-75ac-4222-942e-9ad0604646a4 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:20.255574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 823/1634] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:20 2022] PUT /identity/v3/projects/f2183079411d478587a26c465289e3d7/users/d96944e867914a7e859170cf6f30b61f/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:20.255574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14de50f-4f61-484d-8af8-89109cdb7f97 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:20.354235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14de50f-4f61-484d-8af8-89109cdb7f97 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:20.360638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ec622-4a5b-4857-ab37-96944546f817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:20.361879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ec622-4a5b-4857-ab37-96944546f817 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:20.362543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ec622-4a5b-4857-ab37-96944546f817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:20.363118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ec622-4a5b-4857-ab37-96944546f817 admin admin] PATH_INFO: `/v3/projects/edad36b182f9411084b55f288aa6afa1/users/1ed919ff26a64b47a6f715c0efad85c7/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:20.366653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ec622-4a5b-4857-ab37-96944546f817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edad36b182f9411084b55f288aa6afa1, user_id=1ed919ff26a64b47a6f715c0efad85c7, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:20.370543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ec622-4a5b-4857-ab37-96944546f817 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:20.396575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 824/1635] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:20 2022] PUT /identity/v3/projects/edad36b182f9411084b55f288aa6afa1/users/1ed919ff26a64b47a6f715c0efad85c7/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:20.396575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccbb3a-1d61-4df5-8567-7737d560805c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:20.396575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccbb3a-1d61-4df5-8567-7737d560805c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:20.396575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccbb3a-1d61-4df5-8567-7737d560805c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:20.472961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0c23b-a559-46ce-add6-a0be35fb394b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:20.475211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0c23b-a559-46ce-add6-a0be35fb394b admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:20.475720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0c23b-a559-46ce-add6-a0be35fb394b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:20.476205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0c23b-a559-46ce-add6-a0be35fb394b admin admin] PATH_INFO: `/v3/projects/ef1382625553428eb1f818c507cafa7e/users/a37f484345f34966b28d5501cb9b86a6/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:20.483147 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0c23b-a559-46ce-add6-a0be35fb394b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1382625553428eb1f818c507cafa7e, user_id=a37f484345f34966b28d5501cb9b86a6, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:20.486463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0c23b-a559-46ce-add6-a0be35fb394b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:20.511412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-82ccbb3a-1d61-4df5-8567-7737d560805c None None] MFA Rules not processed for user `d96944e867914a7e859170cf6f30b61f`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:20.514126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-82ccbb3a-1d61-4df5-8567-7737d560805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:20.542777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 812/1636] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:20 2022] PUT /identity/v3/projects/ef1382625553428eb1f818c507cafa7e/users/a37f484345f34966b28d5501cb9b86a6/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:20.544699 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a920975-627f-45a1-9999-793fa1a76adc None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:20.553776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a920975-627f-45a1-9999-793fa1a76adc None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:20.554240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a920975-627f-45a1-9999-793fa1a76adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:20.646134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0a920975-627f-45a1-9999-793fa1a76adc None None] MFA Rules not processed for user `1ed919ff26a64b47a6f715c0efad85c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:20.647470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0a920975-627f-45a1-9999-793fa1a76adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:20.767851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 825/1637] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:20.774600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380ec3d-a36d-4fcb-a8cc-3f56e1fad847 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:20.774891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380ec3d-a36d-4fcb-a8cc-3f56e1fad847 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:20.775685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380ec3d-a36d-4fcb-a8cc-3f56e1fad847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:20.867514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 813/1638] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:20.868635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a920975-627f-45a1-9999-793fa1a76adc None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:20.885266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f380ec3d-a36d-4fcb-a8cc-3f56e1fad847 None None] MFA Rules not processed for user `a37f484345f34966b28d5501cb9b86a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:20.891618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f380ec3d-a36d-4fcb-a8cc-3f56e1fad847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:20.983453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a920975-627f-45a1-9999-793fa1a76adc None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:20.995417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328144c-13af-420f-801a-4cad8822227e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:20.996169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328144c-13af-420f-801a-4cad8822227e service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:20.996517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328144c-13af-420f-801a-4cad8822227e service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:20.996789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328144c-13af-420f-801a-4cad8822227e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:21.015651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328144c-13af-420f-801a-4cad8822227e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:21.022900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328144c-13af-420f-801a-4cad8822227e service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:21.052417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 814/1639] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:21.053416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328144c-13af-420f-801a-4cad8822227e service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:21.069873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328144c-13af-420f-801a-4cad8822227e service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:21.075220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:21.076711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36affedf-8c53-488b-9144-e6156d36971a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:21.079072 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36affedf-8c53-488b-9144-e6156d36971a service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:21.079481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36affedf-8c53-488b-9144-e6156d36971a service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:21.081485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36affedf-8c53-488b-9144-e6156d36971a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:21.095542 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:21.096670 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 826/1640] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:21.106534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36affedf-8c53-488b-9144-e6156d36971a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:21.108083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36affedf-8c53-488b-9144-e6156d36971a service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:21.118493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cf1f0-ee85-4228-bef7-170f1eaf0ca6 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:21.118773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cf1f0-ee85-4228-bef7-170f1eaf0ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:21.119042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cf1f0-ee85-4228-bef7-170f1eaf0ca6 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:21.120757 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 827/1641] 10.222.0.60 () {54 vars in 965 bytes} [Sat Sep 17 04:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:28:21.135196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d22f4-225f-43b8-8a7c-2dc4d0e4c676 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:21.135509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d22f4-225f-43b8-8a7c-2dc4d0e4c676 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:21.135764 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d22f4-225f-43b8-8a7c-2dc4d0e4c676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:21.150177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 815/1642] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:21.165236 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c65d22f4-225f-43b8-8a7c-2dc4d0e4c676 None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:21.166461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c65d22f4-225f-43b8-8a7c-2dc4d0e4c676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:21.179337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36affedf-8c53-488b-9144-e6156d36971a service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:21.188860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36affedf-8c53-488b-9144-e6156d36971a service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:21.206555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d31550-aa41-4884-86e2-712ee70f8cc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:21.206989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d31550-aa41-4884-86e2-712ee70f8cc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:21.207267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d31550-aa41-4884-86e2-712ee70f8cc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:21.207539 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d31550-aa41-4884-86e2-712ee70f8cc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:21.234399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d31550-aa41-4884-86e2-712ee70f8cc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:21.235886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d31550-aa41-4884-86e2-712ee70f8cc6 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:21.275490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 816/1643] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:28:21.312430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 828/1644] 10.222.0.60 () {58 vars in 1062 bytes} [Sat Sep 17 04:28:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:28:21.331339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e676a1-e5b5-49f2-bd25-94af0bde0aa8 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:21.331622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e676a1-e5b5-49f2-bd25-94af0bde0aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:21.331913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e676a1-e5b5-49f2-bd25-94af0bde0aa8 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:21.341263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 817/1645] 10.222.0.60 () {54 vars in 912 bytes} [Sat Sep 17 04:28:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:28:21.342633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f56e9-5e2c-4736-9d70-9aa273ac9ee1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:21.342910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f56e9-5e2c-4736-9d70-9aa273ac9ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:21.343185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f56e9-5e2c-4736-9d70-9aa273ac9ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:21.423881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a29f56e9-5e2c-4736-9d70-9aa273ac9ee1 None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:21.425141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a29f56e9-5e2c-4736-9d70-9aa273ac9ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:21.459531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 829/1646] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:28:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:28:21.582324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e676a1-e5b5-49f2-bd25-94af0bde0aa8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:21.602506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e676a1-e5b5-49f2-bd25-94af0bde0aa8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:21.612213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d885ba8-0af2-4edb-809e-5a29ed648853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:21.612854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d885ba8-0af2-4edb-809e-5a29ed648853 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:21.613154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d885ba8-0af2-4edb-809e-5a29ed648853 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:21.613443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d885ba8-0af2-4edb-809e-5a29ed648853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:21.638748 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d885ba8-0af2-4edb-809e-5a29ed648853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:21.640397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d885ba8-0af2-4edb-809e-5a29ed648853 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:21.676712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 818/1647] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:21.698751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f56e9-5e2c-4736-9d70-9aa273ac9ee1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:21.712628 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f56e9-5e2c-4736-9d70-9aa273ac9ee1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:21.717721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755d115-e45b-4d99-9d24-5b1e38094d77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:21.718534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755d115-e45b-4d99-9d24-5b1e38094d77 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:21.768320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755d115-e45b-4d99-9d24-5b1e38094d77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:21.768320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755d115-e45b-4d99-9d24-5b1e38094d77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:21.869429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800da89d-59c0-4e0b-adc4-41fc4453d555 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:21.870419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800da89d-59c0-4e0b-adc4-41fc4453d555 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:21.870827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800da89d-59c0-4e0b-adc4-41fc4453d555 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:21.874864 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 819/1648] 10.222.0.60 () {54 vars in 912 bytes} [Sat Sep 17 04:28:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:28:21.895169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa08d4-8114-4ffb-8fa5-488e878ad60a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:21.896768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa08d4-8114-4ffb-8fa5-488e878ad60a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:21.897222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa08d4-8114-4ffb-8fa5-488e878ad60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:21.918117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755d115-e45b-4d99-9d24-5b1e38094d77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:21.926943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755d115-e45b-4d99-9d24-5b1e38094d77 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:21.939406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0bfa08d4-8114-4ffb-8fa5-488e878ad60a None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:21.941025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0bfa08d4-8114-4ffb-8fa5-488e878ad60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:21.943511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 830/1649] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:21.962913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 820/1650] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:28:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:22.329249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755d115-e45b-4d99-9d24-5b1e38094d77 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:22.349870 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755d115-e45b-4d99-9d24-5b1e38094d77 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:22.364368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ba932-1a39-4314-bc21-299dcd494829 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:22.369541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ba932-1a39-4314-bc21-299dcd494829 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:22.369719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ba932-1a39-4314-bc21-299dcd494829 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:22.369993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ba932-1a39-4314-bc21-299dcd494829 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:22.386682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ba932-1a39-4314-bc21-299dcd494829 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:22.389429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ba932-1a39-4314-bc21-299dcd494829 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:22.426229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 831/1651] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:22.546772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa08d4-8114-4ffb-8fa5-488e878ad60a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:22.574629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa08d4-8114-4ffb-8fa5-488e878ad60a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:22.588560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f3c71-2ecb-454d-99e1-1344fbdeba33 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd96944e867914a7e859170cf6f30b61f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2183079411d478587a26c465289e3d7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:22.592647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f3c71-2ecb-454d-99e1-1344fbdeba33 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:22.593109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f3c71-2ecb-454d-99e1-1344fbdeba33 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:22.593545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f3c71-2ecb-454d-99e1-1344fbdeba33 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:22.596906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f3c71-2ecb-454d-99e1-1344fbdeba33 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:22.597389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f3c71-2ecb-454d-99e1-1344fbdeba33 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:22.599165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f3c71-2ecb-454d-99e1-1344fbdeba33 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:22.635411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 821/1652] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:22.638702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ba932-1a39-4314-bc21-299dcd494829 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:22.657189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ba932-1a39-4314-bc21-299dcd494829 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:22.688839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a04e64-ee3b-4380-b524-59f66c4e340a tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd96944e867914a7e859170cf6f30b61f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2183079411d478587a26c465289e3d7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:22.689703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a04e64-ee3b-4380-b524-59f66c4e340a tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:22.690123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a04e64-ee3b-4380-b524-59f66c4e340a tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:22.690524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a04e64-ee3b-4380-b524-59f66c4e340a tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:22.691331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a04e64-ee3b-4380-b524-59f66c4e340a tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:22.693453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a04e64-ee3b-4380-b524-59f66c4e340a tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:22.736108 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f3c71-2ecb-454d-99e1-1344fbdeba33 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:22.771148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 832/1653] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:22 2022] POST /identity/v3/projects => generated 354 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:22.774892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a04e64-ee3b-4380-b524-59f66c4e340a tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:22.883494 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f3c71-2ecb-454d-99e1-1344fbdeba33 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:22.957568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:23.007495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a04e64-ee3b-4380-b524-59f66c4e340a tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:23.016748 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:23.017598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:23.018080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:23.018853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] PATH_INFO: `/v3/users/8a4732d60b5441d8bd2f0896d64ca832` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:23.021259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6edd9-ca58-4512-b453-9ed623bbb8d5 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd96944e867914a7e859170cf6f30b61f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2183079411d478587a26c465289e3d7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:23.021899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6edd9-ca58-4512-b453-9ed623bbb8d5 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:23.022182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6edd9-ca58-4512-b453-9ed623bbb8d5 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:23.022284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a4732d60b5441d8bd2f0896d64ca832)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:23.022456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6edd9-ca58-4512-b453-9ed623bbb8d5 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:23.023125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6edd9-ca58-4512-b453-9ed623bbb8d5 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:23.023429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6edd9-ca58-4512-b453-9ed623bbb8d5 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:23.025100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6edd9-ca58-4512-b453-9ed623bbb8d5 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:23.025215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:28:23.025215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 17 04:28:23.025215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: Sep 17 04:28:23.026671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:23.070945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 833/1654] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:28:23.176709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a4732d60b5441d8bd2f0896d64ca832'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:23.195994 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a4732d60b5441d8bd2f0896d64ca832'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:23.197244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a4732d60b5441d8bd2f0896d64ca832'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:23.202741 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:23.220666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a4732d60b5441d8bd2f0896d64ca832'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:23.260893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a4732d60b5441d8bd2f0896d64ca832'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:23.273213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 822/1655] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:22 2022] DELETE /identity/v3/users/8a4732d60b5441d8bd2f0896d64ca832 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:28:23.278097 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4174d-2521-4316-ad30-98dd84642a5b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:23.278369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4174d-2521-4316-ad30-98dd84642a5b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:23.278646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4174d-2521-4316-ad30-98dd84642a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:23.333974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-afa4174d-2521-4316-ad30-98dd84642a5b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:23.335861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-afa4174d-2521-4316-ad30-98dd84642a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:23.532502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 834/1656] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:23.856636 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:23.862410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:23.900382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4174d-2521-4316-ad30-98dd84642a5b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:23.901999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-afa4174d-2521-4316-ad30-98dd84642a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:24.117319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc30340-7d51-4e18-9499-94596a66f0df admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:24.128779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945d4b8-a65a-487e-8f2b-18a020b5222b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:24.129944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d4b8-a65a-487e-8f2b-18a020b5222b service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:24.130219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d4b8-a65a-487e-8f2b-18a020b5222b service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:24.130533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d4b8-a65a-487e-8f2b-18a020b5222b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:24.148299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945d4b8-a65a-487e-8f2b-18a020b5222b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:24.154650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945d4b8-a65a-487e-8f2b-18a020b5222b service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:24.160444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4174d-2521-4316-ad30-98dd84642a5b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:24.169837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd96944e867914a7e859170cf6f30b61f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2183079411d478587a26c465289e3d7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:24.170725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:24.170840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:24.171325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] PATH_INFO: `/v3/projects/9c6337fe541b404c9ce4e01b310c57de` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:24.173802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9c6337fe541b404c9ce4e01b310c57de)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:24.174847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 04:28:24.174847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 17 04:28:24.174847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: Sep 17 04:28:24.176035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:24.178310 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 823/1657] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:24.326129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c6337fe541b404c9ce4e01b310c57de', 'request_id': 'req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:24.356154 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c6337fe541b404c9ce4e01b310c57de', 'request_id': 'req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:24.391936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c6337fe541b404c9ce4e01b310c57de', 'request_id': 'req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:24.440634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] The token cache is being invalidate because project 9c6337fe541b404c9ce4e01b310c57de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:24.440926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:24.441805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-495398ad-69fb-4c02-8b7d-bddb4d7efcb0 tempest-VolumeQuotaClassesTest-1183478716 tempest-VolumeQuotaClassesTest-1183478716-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:24.447928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 835/1658] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:23 2022] DELETE /identity/v3/projects/9c6337fe541b404c9ce4e01b310c57de => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:24.962481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945d4b8-a65a-487e-8f2b-18a020b5222b service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:24.964718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1945d4b8-a65a-487e-8f2b-18a020b5222b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:24.982782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945d4b8-a65a-487e-8f2b-18a020b5222b service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:25.111535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c178a7-38ac-4db1-953a-5b35075a8567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:25.112374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c178a7-38ac-4db1-953a-5b35075a8567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:25.112975 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c178a7-38ac-4db1-953a-5b35075a8567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:25.113407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c178a7-38ac-4db1-953a-5b35075a8567 admin admin] PATH_INFO: `/v3/users/c520e361f89545cea9b84b0a349a3462` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:25.117653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c178a7-38ac-4db1-953a-5b35075a8567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c520e361f89545cea9b84b0a349a3462)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:25.119772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c178a7-38ac-4db1-953a-5b35075a8567 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:25.207269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-47c178a7-38ac-4db1-953a-5b35075a8567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c520e361f89545cea9b84b0a349a3462'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:25.238535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-47c178a7-38ac-4db1-953a-5b35075a8567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c520e361f89545cea9b84b0a349a3462'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:25.254619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-47c178a7-38ac-4db1-953a-5b35075a8567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c520e361f89545cea9b84b0a349a3462'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:25.286953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-47c178a7-38ac-4db1-953a-5b35075a8567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c520e361f89545cea9b84b0a349a3462'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:25.336280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:25.340559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-47c178a7-38ac-4db1-953a-5b35075a8567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c520e361f89545cea9b84b0a349a3462'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:25.359044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 824/1659] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:24 2022] DELETE /identity/v3/users/c520e361f89545cea9b84b0a349a3462 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:25.365496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9d2b1-281e-4954-ac47-5e938b511296 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:25.366401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9d2b1-281e-4954-ac47-5e938b511296 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:25.366835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9d2b1-281e-4954-ac47-5e938b511296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:25.413388 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a5e9d2b1-281e-4954-ac47-5e938b511296 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:25.417071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a5e9d2b1-281e-4954-ac47-5e938b511296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:25.436036 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:25.545768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b8006-ece9-4eea-b1de-7920f0b0087a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:25.546172 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b8006-ece9-4eea-b1de-7920f0b0087a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:25.546843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b8006-ece9-4eea-b1de-7920f0b0087a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:25.644167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-bb4b8006-ece9-4eea-b1de-7920f0b0087a None None] MFA Rules not processed for user `094abcdcd1f54a658564357056e7849a`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:25.646040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 836/1660] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:25.646416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bb4b8006-ece9-4eea-b1de-7920f0b0087a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:25.682123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9d2b1-281e-4954-ac47-5e938b511296 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:25.684975 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a5e9d2b1-281e-4954-ac47-5e938b511296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:25.843949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9d2b1-281e-4954-ac47-5e938b511296 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:25.851996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e90ee-4faf-4ef0-ba58-58aa1cf69419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:25.861526 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 825/1661] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:25.864120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e90ee-4faf-4ef0-ba58-58aa1cf69419 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:25.864416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e90ee-4faf-4ef0-ba58-58aa1cf69419 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:25.864695 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e90ee-4faf-4ef0-ba58-58aa1cf69419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:25.871295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b8006-ece9-4eea-b1de-7920f0b0087a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:25.872969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bb4b8006-ece9-4eea-b1de-7920f0b0087a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:25.888928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e90ee-4faf-4ef0-ba58-58aa1cf69419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:25.890499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e90ee-4faf-4ef0-ba58-58aa1cf69419 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:25.926088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 837/1662] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:28:26.093607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b8006-ece9-4eea-b1de-7920f0b0087a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:26.105844 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457883d-368f-4272-96a2-3203c1e44f20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:26.107282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457883d-368f-4272-96a2-3203c1e44f20 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:26.107638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457883d-368f-4272-96a2-3203c1e44f20 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:26.107981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457883d-368f-4272-96a2-3203c1e44f20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:26.127375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457883d-368f-4272-96a2-3203c1e44f20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:26.130805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457883d-368f-4272-96a2-3203c1e44f20 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:26.146225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e90ee-4faf-4ef0-ba58-58aa1cf69419 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:26.148120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-411e90ee-4faf-4ef0-ba58-58aa1cf69419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:26.165269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 826/1663] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:26.197138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e90ee-4faf-4ef0-ba58-58aa1cf69419 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:26.247749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:26.261890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:26.265192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:26.265491 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] PATH_INFO: `/v3/projects/8789a927bb6442da93e546b74955f415` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:26.272800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8789a927bb6442da93e546b74955f415)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:26.284598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:26.424905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8789a927bb6442da93e546b74955f415', 'request_id': 'req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:26.462335 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8789a927bb6442da93e546b74955f415', 'request_id': 'req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:26.471311 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8789a927bb6442da93e546b74955f415', 'request_id': 'req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:26.505685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] The token cache is being invalidate because project 8789a927bb6442da93e546b74955f415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:26.506331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:26.509238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:26.517904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 838/1664] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:26 2022] DELETE /identity/v3/projects/8789a927bb6442da93e546b74955f415 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:26.524903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5a376-7587-4475-8f64-493fd124dd36 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:26.525525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5a376-7587-4475-8f64-493fd124dd36 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:26.526104 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5a376-7587-4475-8f64-493fd124dd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:26.536147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:26.538636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:26.577601 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cdd5a376-7587-4475-8f64-493fd124dd36 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:26.579518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cdd5a376-7587-4475-8f64-493fd124dd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:26.680908 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:26.779662 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd7a93-1a9c-468f-aaea-37e9bf2149f1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:26.811387 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d41a5a-f1f9-4178-a68e-37504667b332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:26.817682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d41a5a-f1f9-4178-a68e-37504667b332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:26.817967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d41a5a-f1f9-4178-a68e-37504667b332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:26.818243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d41a5a-f1f9-4178-a68e-37504667b332 admin admin] PATH_INFO: `/v3/users/4742f609c089410c952235906ad98bcd` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:26.819849 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d41a5a-f1f9-4178-a68e-37504667b332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4742f609c089410c952235906ad98bcd)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:26.822744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d41a5a-f1f9-4178-a68e-37504667b332 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:26.856308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 827/1665] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:26.857903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac656c55-6390-4ad8-8bd0-0358214c1715 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:26.858193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac656c55-6390-4ad8-8bd0-0358214c1715 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:26.858518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac656c55-6390-4ad8-8bd0-0358214c1715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:26.927399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ac656c55-6390-4ad8-8bd0-0358214c1715 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:26.930026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ac656c55-6390-4ad8-8bd0-0358214c1715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:26.936549 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e6d41a5a-f1f9-4178-a68e-37504667b332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4742f609c089410c952235906ad98bcd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:26.957244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e6d41a5a-f1f9-4178-a68e-37504667b332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4742f609c089410c952235906ad98bcd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:26.958659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e6d41a5a-f1f9-4178-a68e-37504667b332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4742f609c089410c952235906ad98bcd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:26.985970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e6d41a5a-f1f9-4178-a68e-37504667b332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4742f609c089410c952235906ad98bcd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:26.987796 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 828/1666] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:26.988795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac656c55-6390-4ad8-8bd0-0358214c1715 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:27.026655 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e6d41a5a-f1f9-4178-a68e-37504667b332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4742f609c089410c952235906ad98bcd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:27.029537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:27.040416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac656c55-6390-4ad8-8bd0-0358214c1715 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:27.045270 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 839/1667] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:26 2022] DELETE /identity/v3/users/4742f609c089410c952235906ad98bcd => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:28:27.047700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baf822-40a4-406e-b5e4-4319b19b3567 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:27.048047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baf822-40a4-406e-b5e4-4319b19b3567 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:27.054308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baf822-40a4-406e-b5e4-4319b19b3567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:27.152648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-02baf822-40a4-406e-b5e4-4319b19b3567 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:27.154465 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-02baf822-40a4-406e-b5e4-4319b19b3567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:27.329824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6076e02-0e61-46b5-af7f-83d39efbf043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:27.330459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6076e02-0e61-46b5-af7f-83d39efbf043 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:27.330740 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6076e02-0e61-46b5-af7f-83d39efbf043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:27.331696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6076e02-0e61-46b5-af7f-83d39efbf043 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:27.332368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6076e02-0e61-46b5-af7f-83d39efbf043 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:27.332672 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6076e02-0e61-46b5-af7f-83d39efbf043 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:27.334379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6076e02-0e61-46b5-af7f-83d39efbf043 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:27.394224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 840/1668] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:27.396130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 829/1669] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:27.401229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02baf822-40a4-406e-b5e4-4319b19b3567 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:27.424915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02baf822-40a4-406e-b5e4-4319b19b3567 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:27.429900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b3ab4-6fa8-48af-9a81-55d7d83afb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:27.430531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b3ab4-6fa8-48af-9a81-55d7d83afb97 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:27.431543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b3ab4-6fa8-48af-9a81-55d7d83afb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:27.431826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b3ab4-6fa8-48af-9a81-55d7d83afb97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:27.432692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b3ab4-6fa8-48af-9a81-55d7d83afb97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:27.434816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b3ab4-6fa8-48af-9a81-55d7d83afb97 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:27.480870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6076e02-0e61-46b5-af7f-83d39efbf043 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:27.483158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f6076e02-0e61-46b5-af7f-83d39efbf043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:27.486632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 841/1670] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:27 2022] POST /identity/v3/projects => generated 365 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:27.491328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b3ab4-6fa8-48af-9a81-55d7d83afb97 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:27.625569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6076e02-0e61-46b5-af7f-83d39efbf043 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:27.626761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:27.741126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b3ab4-6fa8-48af-9a81-55d7d83afb97 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:27.753480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba1cd-071b-40f9-bd32-5264325c9530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:27.754130 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba1cd-071b-40f9-bd32-5264325c9530 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:27.754413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba1cd-071b-40f9-bd32-5264325c9530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:27.754701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba1cd-071b-40f9-bd32-5264325c9530 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:27.755314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bba1cd-071b-40f9-bd32-5264325c9530 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:27.763372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bba1cd-071b-40f9-bd32-5264325c9530 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:27.771969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56738f-8243-4ab3-b54f-da63738a31f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:27.772630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56738f-8243-4ab3-b54f-da63738a31f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:27.772907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56738f-8243-4ab3-b54f-da63738a31f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:27.773204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56738f-8243-4ab3-b54f-da63738a31f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:27.810957 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56738f-8243-4ab3-b54f-da63738a31f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:27.814142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56738f-8243-4ab3-b54f-da63738a31f6 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:27.815663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 842/1671] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:27 2022] POST /identity/v3/users => generated 283 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:27.819721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba1cd-071b-40f9-bd32-5264325c9530 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:27.845991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba1cd-071b-40f9-bd32-5264325c9530 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:27.865053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 830/1672] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:28:27.865854 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c1360-a5aa-4374-b63d-46f1c919d7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:27.866871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c1360-a5aa-4374-b63d-46f1c919d7af admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:27.866871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c1360-a5aa-4374-b63d-46f1c919d7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:27.867183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c1360-a5aa-4374-b63d-46f1c919d7af admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:27.867680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c1360-a5aa-4374-b63d-46f1c919d7af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:27.869415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c1360-a5aa-4374-b63d-46f1c919d7af admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:27.925021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 843/1673] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:27.928432 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56738f-8243-4ab3-b54f-da63738a31f6 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:27.950191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56738f-8243-4ab3-b54f-da63738a31f6 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:27.989441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76eaab-dc28-4d78-8ab6-59bf5549286b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:27.990332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76eaab-dc28-4d78-8ab6-59bf5549286b admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:27.990798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76eaab-dc28-4d78-8ab6-59bf5549286b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:27.991260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76eaab-dc28-4d78-8ab6-59bf5549286b admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:27.999899 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76eaab-dc28-4d78-8ab6-59bf5549286b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:28.003567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76eaab-dc28-4d78-8ab6-59bf5549286b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:28.021534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 831/1674] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:28.027996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c1360-a5aa-4374-b63d-46f1c919d7af admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:28.042996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c1360-a5aa-4374-b63d-46f1c919d7af admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:28.058162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc618cb-7dfb-43b4-bd33-e856f6da6141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:28.059190 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc618cb-7dfb-43b4-bd33-e856f6da6141 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:28.059853 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc618cb-7dfb-43b4-bd33-e856f6da6141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:28.060325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc618cb-7dfb-43b4-bd33-e856f6da6141 admin admin] PATH_INFO: `/v3/projects/ce667700f6f246f3b67ca6628c67a2c5/users/b09ebfa88b70409fb80f91ce95a21af4/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:28.086620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc618cb-7dfb-43b4-bd33-e856f6da6141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce667700f6f246f3b67ca6628c67a2c5, user_id=b09ebfa88b70409fb80f91ce95a21af4, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:28.093633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc618cb-7dfb-43b4-bd33-e856f6da6141 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:28.127414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 844/1675] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:28 2022] PUT /identity/v3/projects/ce667700f6f246f3b67ca6628c67a2c5/users/b09ebfa88b70409fb80f91ce95a21af4/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:28:28.139602 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55784f-014a-4b03-aabb-61d48b98e4fe None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:28.139890 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55784f-014a-4b03-aabb-61d48b98e4fe None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:28.140288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55784f-014a-4b03-aabb-61d48b98e4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:28.202113 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7f55784f-014a-4b03-aabb-61d48b98e4fe None None] MFA Rules not processed for user `b09ebfa88b70409fb80f91ce95a21af4`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:28.203358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7f55784f-014a-4b03-aabb-61d48b98e4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:28.297790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:28.345188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc618cb-7dfb-43b4-bd33-e856f6da6141 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:28.347182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6cc618cb-7dfb-43b4-bd33-e856f6da6141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:28.384323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 832/1676] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:28.388816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55784f-014a-4b03-aabb-61d48b98e4fe None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:28.554855 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55784f-014a-4b03-aabb-61d48b98e4fe None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:28.568382 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc47064-d82e-48eb-98cc-274c59fde2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:28.572094 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc47064-d82e-48eb-98cc-274c59fde2ef admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:28.572467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc47064-d82e-48eb-98cc-274c59fde2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:28.572619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc47064-d82e-48eb-98cc-274c59fde2ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:28.573452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc47064-d82e-48eb-98cc-274c59fde2ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:28.573702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc618cb-7dfb-43b4-bd33-e856f6da6141 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:28.580507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc47064-d82e-48eb-98cc-274c59fde2ef admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:28.595572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:28.597554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:28.597865 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:28.598134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] PATH_INFO: `/v3/projects/6292f5855c2a437daa4b82fe05ac7eb8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:28.601265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6292f5855c2a437daa4b82fe05ac7eb8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:28.619140 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:28.721640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 833/1677] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:28 2022] POST /identity/v3/projects => generated 363 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:28:28.722757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc47064-d82e-48eb-98cc-274c59fde2ef admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:28.762205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6292f5855c2a437daa4b82fe05ac7eb8', 'request_id': 'req-3aa02d8b-6989-4095-b3c6-7611f5a80573'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:28.800252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6292f5855c2a437daa4b82fe05ac7eb8', 'request_id': 'req-3aa02d8b-6989-4095-b3c6-7611f5a80573'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:28.813446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6292f5855c2a437daa4b82fe05ac7eb8', 'request_id': 'req-3aa02d8b-6989-4095-b3c6-7611f5a80573'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:28.839471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] The token cache is being invalidate because project 6292f5855c2a437daa4b82fe05ac7eb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:28.839773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:28.840611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:28.849032 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 845/1678] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:28 2022] DELETE /identity/v3/projects/6292f5855c2a437daa4b82fe05ac7eb8 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:28:28.852216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:28.854123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:28.925910 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc47064-d82e-48eb-98cc-274c59fde2ef admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:28.950961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:29.043621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f506ed-92b6-48fa-a392-6e6f45882664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:29.044259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f506ed-92b6-48fa-a392-6e6f45882664 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:29.044536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f506ed-92b6-48fa-a392-6e6f45882664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:29.045112 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f506ed-92b6-48fa-a392-6e6f45882664 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:29.046021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f506ed-92b6-48fa-a392-6e6f45882664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:29.049363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f506ed-92b6-48fa-a392-6e6f45882664 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:29.081446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:29.092764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 834/1679] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:28 2022] POST /identity/v3/users => generated 288 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:28:29.097167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f506ed-92b6-48fa-a392-6e6f45882664 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:29.101938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b7f506ed-92b6-48fa-a392-6e6f45882664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:29.124060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa02d8b-6989-4095-b3c6-7611f5a80573 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:29.130305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f506ed-92b6-48fa-a392-6e6f45882664 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:29.137547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:29.138224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:29.138503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:29.139405 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] PATH_INFO: `/v3/users/d96944e867914a7e859170cf6f30b61f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:29.140871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96944e867914a7e859170cf6f30b61f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:29.148839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a49ef8-e084-4a33-b5d3-e0f3f0a6ee2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:29.151425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:29.151790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a49ef8-e084-4a33-b5d3-e0f3f0a6ee2a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:29.152079 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a49ef8-e084-4a33-b5d3-e0f3f0a6ee2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:29.153900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a49ef8-e084-4a33-b5d3-e0f3f0a6ee2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:29.158132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a49ef8-e084-4a33-b5d3-e0f3f0a6ee2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:29.166157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a49ef8-e084-4a33-b5d3-e0f3f0a6ee2a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:29.185196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 835/1680] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:28:29.189385 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a49ef8-e084-4a33-b5d3-e0f3f0a6ee2a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:29.210826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a49ef8-e084-4a33-b5d3-e0f3f0a6ee2a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:29.271493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189684d-4d94-4e56-a78e-87f0ea004904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:29.272997 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189684d-4d94-4e56-a78e-87f0ea004904 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:29.274590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189684d-4d94-4e56-a78e-87f0ea004904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:29.274841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189684d-4d94-4e56-a78e-87f0ea004904 admin admin] PATH_INFO: `/v3/projects/d2969ddbda9b45bf8994696ceba2b564/users/847c984269cf4bcfbd52558e3db96373/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:29.281669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96944e867914a7e859170cf6f30b61f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:29.305136 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96944e867914a7e859170cf6f30b61f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:29.305136 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96944e867914a7e859170cf6f30b61f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:29.317514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96944e867914a7e859170cf6f30b61f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:29.320874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189684d-4d94-4e56-a78e-87f0ea004904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2969ddbda9b45bf8994696ceba2b564, user_id=847c984269cf4bcfbd52558e3db96373, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:29.330212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189684d-4d94-4e56-a78e-87f0ea004904 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:29.342637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96944e867914a7e859170cf6f30b61f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:29.357048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 846/1681] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:28 2022] DELETE /identity/v3/users/d96944e867914a7e859170cf6f30b61f => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:29.391501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 836/1682] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:29 2022] PUT /identity/v3/projects/d2969ddbda9b45bf8994696ceba2b564/users/847c984269cf4bcfbd52558e3db96373/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:29.395579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:29.397357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:29.552899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69900ecf-fe0a-493b-a149-9777a6a661d6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:29.631938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:29.670142 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41938b87-d7f6-4bf6-9df7-abe768fca2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:29.670869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41938b87-d7f6-4bf6-9df7-abe768fca2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:29.671191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41938b87-d7f6-4bf6-9df7-abe768fca2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:29.671719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41938b87-d7f6-4bf6-9df7-abe768fca2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:29.672500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41938b87-d7f6-4bf6-9df7-abe768fca2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:29.681855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41938b87-d7f6-4bf6-9df7-abe768fca2a7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:29.708612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189684d-4d94-4e56-a78e-87f0ea004904 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:29.717103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 847/1683] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:29.723777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189684d-4d94-4e56-a78e-87f0ea004904 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:29.741729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49a309-b7b4-423f-a177-e7bbfce2bbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:29.742444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49a309-b7b4-423f-a177-e7bbfce2bbd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:29.742815 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49a309-b7b4-423f-a177-e7bbfce2bbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:29.743089 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49a309-b7b4-423f-a177-e7bbfce2bbd7 admin admin] PATH_INFO: `/v3/projects/d2969ddbda9b45bf8994696ceba2b564/users/847c984269cf4bcfbd52558e3db96373/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:29.745535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49a309-b7b4-423f-a177-e7bbfce2bbd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2969ddbda9b45bf8994696ceba2b564, user_id=847c984269cf4bcfbd52558e3db96373, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:29.748130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49a309-b7b4-423f-a177-e7bbfce2bbd7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:29.790505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 837/1684] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:29 2022] PUT /identity/v3/projects/d2969ddbda9b45bf8994696ceba2b564/users/847c984269cf4bcfbd52558e3db96373/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:28:29.802248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5aaa4-eec5-476b-bc54-96517b4fddbe None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:29.802248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5aaa4-eec5-476b-bc54-96517b4fddbe None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:29.802248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5aaa4-eec5-476b-bc54-96517b4fddbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:29.806582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9dd4e1-e57e-4b87-8e0b-cf75fdd6bb83 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:29.806859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9dd4e1-e57e-4b87-8e0b-cf75fdd6bb83 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:29.807124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9dd4e1-e57e-4b87-8e0b-cf75fdd6bb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:29.854130 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-81a5aaa4-eec5-476b-bc54-96517b4fddbe None None] MFA Rules not processed for user `847c984269cf4bcfbd52558e3db96373`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:29.855677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-81a5aaa4-eec5-476b-bc54-96517b4fddbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:29.873429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1e9dd4e1-e57e-4b87-8e0b-cf75fdd6bb83 None None] MFA Rules not processed for user `ed4e695113f843fda52ce62792a36bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:29.875062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1e9dd4e1-e57e-4b87-8e0b-cf75fdd6bb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:29.983943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 838/1685] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:29.986700 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9dd4e1-e57e-4b87-8e0b-cf75fdd6bb83 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:30.013478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1e9dd4e1-e57e-4b87-8e0b-cf75fdd6bb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:30.024259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 848/1686] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:30.030826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5aaa4-eec5-476b-bc54-96517b4fddbe None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:30.034189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-81a5aaa4-eec5-476b-bc54-96517b4fddbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:30.165881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9dd4e1-e57e-4b87-8e0b-cf75fdd6bb83 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:30.183342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:30.184369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:30.184652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:30.184930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] PATH_INFO: `/v3/projects/901517d15c4b4c3bae31c3389ff50301` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:30.206754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=901517d15c4b4c3bae31c3389ff50301)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:30.213782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:30.216605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5aaa4-eec5-476b-bc54-96517b4fddbe None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:30.230611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6db0d5-3bea-43bb-ad9c-ea430dddc18d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:30.231522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6db0d5-3bea-43bb-ad9c-ea430dddc18d service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:30.231735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6db0d5-3bea-43bb-ad9c-ea430dddc18d service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:30.231866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6db0d5-3bea-43bb-ad9c-ea430dddc18d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:30.253293 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6db0d5-3bea-43bb-ad9c-ea430dddc18d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:30.255286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6db0d5-3bea-43bb-ad9c-ea430dddc18d service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:30.276642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 849/1687] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:30.277590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6db0d5-3bea-43bb-ad9c-ea430dddc18d service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:30.281124 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6a6db0d5-3bea-43bb-ad9c-ea430dddc18d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:30.373960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:30.391394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901517d15c4b4c3bae31c3389ff50301', 'request_id': 'req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.406067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6db0d5-3bea-43bb-ad9c-ea430dddc18d service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:30.413906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:30.414877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901517d15c4b4c3bae31c3389ff50301', 'request_id': 'req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.421943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901517d15c4b4c3bae31c3389ff50301', 'request_id': 'req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.454379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] The token cache is being invalidate because project 901517d15c4b4c3bae31c3389ff50301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:30.454840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.455719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.463401 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 839/1688] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:29 2022] DELETE /identity/v3/projects/901517d15c4b4c3bae31c3389ff50301 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:30.464785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:30.467662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:30.539311 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44f407-fb12-48f2-aa06-20be58a69422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:30.540091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44f407-fb12-48f2-aa06-20be58a69422 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:30.540439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44f407-fb12-48f2-aa06-20be58a69422 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:30.540782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44f407-fb12-48f2-aa06-20be58a69422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:30.550509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1f44f407-fb12-48f2-aa06-20be58a69422 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:30.627178 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f6b37-e53e-4501-b689-0d4c7bb3c8b3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:30.644855 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b4a55-1d43-4afd-9867-e6a1ec644105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:30.645907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b4a55-1d43-4afd-9867-e6a1ec644105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:30.646352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b4a55-1d43-4afd-9867-e6a1ec644105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:30.646801 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b4a55-1d43-4afd-9867-e6a1ec644105 admin admin] PATH_INFO: `/v3/users/1ed919ff26a64b47a6f715c0efad85c7` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:30.648383 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b4a55-1d43-4afd-9867-e6a1ec644105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ed919ff26a64b47a6f715c0efad85c7)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:30.650716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b4a55-1d43-4afd-9867-e6a1ec644105 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:30.703377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44f407-fb12-48f2-aa06-20be58a69422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:30.705400 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44f407-fb12-48f2-aa06-20be58a69422 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:30.725874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 850/1689] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:30.727113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44f407-fb12-48f2-aa06-20be58a69422 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:30.729965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1f44f407-fb12-48f2-aa06-20be58a69422 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:30.746873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44f407-fb12-48f2-aa06-20be58a69422 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:30.756368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:30.757099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:30.758686 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:30.762102 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3 admin admin] PATH_INFO: `/v3/projects/f2183079411d478587a26c465289e3d7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:30.767554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2183079411d478587a26c465289e3d7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:30.774762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:30.793509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a48b4a55-1d43-4afd-9867-e6a1ec644105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ed919ff26a64b47a6f715c0efad85c7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.816959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a48b4a55-1d43-4afd-9867-e6a1ec644105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ed919ff26a64b47a6f715c0efad85c7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.820386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a48b4a55-1d43-4afd-9867-e6a1ec644105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ed919ff26a64b47a6f715c0efad85c7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.836206 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a48b4a55-1d43-4afd-9867-e6a1ec644105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ed919ff26a64b47a6f715c0efad85c7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.889156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a48b4a55-1d43-4afd-9867-e6a1ec644105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ed919ff26a64b47a6f715c0efad85c7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.914114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 840/1690] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:30 2022] DELETE /identity/v3/users/1ed919ff26a64b47a6f715c0efad85c7 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:30.938279 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2183079411d478587a26c465289e3d7', 'request_id': 'req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.958531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2183079411d478587a26c465289e3d7', 'request_id': 'req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.967138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2183079411d478587a26c465289e3d7', 'request_id': 'req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.990623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3 admin admin] The token cache is being invalidate because project f2183079411d478587a26c465289e3d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:30.991131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.992931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9ddd7e2b-989d-44e6-ab34-9d0b2c43f5c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:30.999486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 851/1691] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:30 2022] DELETE /identity/v3/projects/f2183079411d478587a26c465289e3d7 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:31.005654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0f4d2-e9ec-4e00-b6cf-dfe1584a4286 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:31.005941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0f4d2-e9ec-4e00-b6cf-dfe1584a4286 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:31.006217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0f4d2-e9ec-4e00-b6cf-dfe1584a4286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:31.039173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-85f0f4d2-e9ec-4e00-b6cf-dfe1584a4286 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:31.040981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-85f0f4d2-e9ec-4e00-b6cf-dfe1584a4286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:31.151505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:31.202547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 841/1692] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:28:31.211859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d2798-1af8-4670-8d86-361e3d2a49b8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:31.212483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d2798-1af8-4670-8d86-361e3d2a49b8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:31.212768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d2798-1af8-4670-8d86-361e3d2a49b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:31.304636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-924d2798-1af8-4670-8d86-361e3d2a49b8 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:31.315406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-924d2798-1af8-4670-8d86-361e3d2a49b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:31.343615 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 852/1693] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:31.349837 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0f4d2-e9ec-4e00-b6cf-dfe1584a4286 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:31.366150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0f4d2-e9ec-4e00-b6cf-dfe1584a4286 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:31.376173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dec5b-7dd3-4a76-8f03-4c83703225b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:31.376969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dec5b-7dd3-4a76-8f03-4c83703225b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:31.384309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dec5b-7dd3-4a76-8f03-4c83703225b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:31.385280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dec5b-7dd3-4a76-8f03-4c83703225b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:31.397599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dec5b-7dd3-4a76-8f03-4c83703225b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:31.397599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dec5b-7dd3-4a76-8f03-4c83703225b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:31.401783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dec5b-7dd3-4a76-8f03-4c83703225b6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:31.436691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 842/1694] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:31.441362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d2798-1af8-4670-8d86-361e3d2a49b8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:31.460870 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d2798-1af8-4670-8d86-361e3d2a49b8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:31.485971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee9ea5-5f37-402c-b4fd-7025f5cf26ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:31.486772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee9ea5-5f37-402c-b4fd-7025f5cf26ca admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:31.487056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee9ea5-5f37-402c-b4fd-7025f5cf26ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:31.487332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee9ea5-5f37-402c-b4fd-7025f5cf26ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:31.487976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee9ea5-5f37-402c-b4fd-7025f5cf26ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:31.489941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee9ea5-5f37-402c-b4fd-7025f5cf26ca admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:31.599645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 853/1695] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:31 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:31.603421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dec5b-7dd3-4a76-8f03-4c83703225b6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:31.621371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dec5b-7dd3-4a76-8f03-4c83703225b6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:31.817838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d1583-96e6-4c89-bcea-99ef4201e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:31.818475 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d1583-96e6-4c89-bcea-99ef4201e472 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:31.818752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d1583-96e6-4c89-bcea-99ef4201e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:31.819023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d1583-96e6-4c89-bcea-99ef4201e472 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:31.819672 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d1583-96e6-4c89-bcea-99ef4201e472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:31.821680 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d1583-96e6-4c89-bcea-99ef4201e472 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:31.894522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 843/1696] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:31 2022] POST /identity/v3/users => generated 294 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:31.899097 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee9ea5-5f37-402c-b4fd-7025f5cf26ca admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:31.920304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee9ea5-5f37-402c-b4fd-7025f5cf26ca admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:31.927294 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4858ca-404f-4e8f-a13a-9ed4c4282325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:31.928148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4858ca-404f-4e8f-a13a-9ed4c4282325 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:31.928629 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4858ca-404f-4e8f-a13a-9ed4c4282325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:31.929066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4858ca-404f-4e8f-a13a-9ed4c4282325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:31.937441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d1583-96e6-4c89-bcea-99ef4201e472 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:31.939940 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5f3d1583-96e6-4c89-bcea-99ef4201e472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:31.947710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4858ca-404f-4e8f-a13a-9ed4c4282325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:31.950744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4858ca-404f-4e8f-a13a-9ed4c4282325 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:31.955755 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d1583-96e6-4c89-bcea-99ef4201e472 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:31.971979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:31.984712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627849fc-8171-4e92-8d4a-63a0128d4d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:31.992359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627849fc-8171-4e92-8d4a-63a0128d4d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:31.992795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627849fc-8171-4e92-8d4a-63a0128d4d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:31.993211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627849fc-8171-4e92-8d4a-63a0128d4d57 admin admin] PATH_INFO: `/v3/users/b09ebfa88b70409fb80f91ce95a21af4` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:31.995476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627849fc-8171-4e92-8d4a-63a0128d4d57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b09ebfa88b70409fb80f91ce95a21af4)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:31.996607 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 854/1697] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:32.017941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4858ca-404f-4e8f-a13a-9ed4c4282325 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:32.019218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627849fc-8171-4e92-8d4a-63a0128d4d57 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:32.056972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4858ca-404f-4e8f-a13a-9ed4c4282325 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:32.066650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d3058-68b7-4afa-a4f6-0ef11aeb3cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:32.067452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d3058-68b7-4afa-a4f6-0ef11aeb3cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:32.068339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d3058-68b7-4afa-a4f6-0ef11aeb3cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:32.071515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d3058-68b7-4afa-a4f6-0ef11aeb3cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:32.071515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d3058-68b7-4afa-a4f6-0ef11aeb3cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:32.073320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d3058-68b7-4afa-a4f6-0ef11aeb3cb2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:32.095286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 855/1698] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:32.096988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d3058-68b7-4afa-a4f6-0ef11aeb3cb2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:32.126976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d3058-68b7-4afa-a4f6-0ef11aeb3cb2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:32.138209 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-627849fc-8171-4e92-8d4a-63a0128d4d57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b09ebfa88b70409fb80f91ce95a21af4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:32.145000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace6f52-f996-4d1d-ad0e-b86d225f5908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:32.150250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace6f52-f996-4d1d-ad0e-b86d225f5908 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:32.150654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace6f52-f996-4d1d-ad0e-b86d225f5908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:32.151806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace6f52-f996-4d1d-ad0e-b86d225f5908 admin admin] PATH_INFO: `/v3/projects/e6d64dd538004d0a88095f0b5b0f554d/users/16237c4c3d574a5393476f7e078c652c/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:32.166737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-627849fc-8171-4e92-8d4a-63a0128d4d57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b09ebfa88b70409fb80f91ce95a21af4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:32.167615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-627849fc-8171-4e92-8d4a-63a0128d4d57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b09ebfa88b70409fb80f91ce95a21af4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:32.185010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-627849fc-8171-4e92-8d4a-63a0128d4d57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b09ebfa88b70409fb80f91ce95a21af4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:32.205886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace6f52-f996-4d1d-ad0e-b86d225f5908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d64dd538004d0a88095f0b5b0f554d, user_id=16237c4c3d574a5393476f7e078c652c, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:32.213280 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace6f52-f996-4d1d-ad0e-b86d225f5908 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:32.218612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-627849fc-8171-4e92-8d4a-63a0128d4d57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b09ebfa88b70409fb80f91ce95a21af4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:32.233459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 844/1699] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:31 2022] DELETE /identity/v3/users/b09ebfa88b70409fb80f91ce95a21af4 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:28:32.237074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281f69-1d31-40b6-8c30-24ca49690f78 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:32.246700 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281f69-1d31-40b6-8c30-24ca49690f78 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:32.247301 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281f69-1d31-40b6-8c30-24ca49690f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:32.257654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 856/1700] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:32 2022] PUT /identity/v3/projects/e6d64dd538004d0a88095f0b5b0f554d/users/16237c4c3d574a5393476f7e078c652c/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:32.257654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace6f52-f996-4d1d-ad0e-b86d225f5908 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:32.257654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1ace6f52-f996-4d1d-ad0e-b86d225f5908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:32.287365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6f281f69-1d31-40b6-8c30-24ca49690f78 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:32.289933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6f281f69-1d31-40b6-8c30-24ca49690f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:32.336658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:32.404935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 845/1701] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:32.406542 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6cbb4-b588-40c7-b506-1de96b2858ec None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:32.406833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6cbb4-b588-40c7-b506-1de96b2858ec None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:32.407209 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6cbb4-b588-40c7-b506-1de96b2858ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:32.470924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-48c6cbb4-b588-40c7-b506-1de96b2858ec None None] MFA Rules not processed for user `16237c4c3d574a5393476f7e078c652c`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:32.472825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-48c6cbb4-b588-40c7-b506-1de96b2858ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:32.504010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace6f52-f996-4d1d-ad0e-b86d225f5908 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:32.516584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a65380-cbb3-4f32-8622-267d246018b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:32.518993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a65380-cbb3-4f32-8622-267d246018b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:32.521411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a65380-cbb3-4f32-8622-267d246018b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:32.522527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a65380-cbb3-4f32-8622-267d246018b0 admin admin] PATH_INFO: `/v3/projects/edad36b182f9411084b55f288aa6afa1` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:32.527169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a65380-cbb3-4f32-8622-267d246018b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edad36b182f9411084b55f288aa6afa1)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:32.531545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a65380-cbb3-4f32-8622-267d246018b0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:32.677282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 846/1702] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:32.678529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6cbb4-b588-40c7-b506-1de96b2858ec None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:32.685607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-48c6cbb4-b588-40c7-b506-1de96b2858ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:32.687454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-05a65380-cbb3-4f32-8622-267d246018b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edad36b182f9411084b55f288aa6afa1', 'request_id': 'req-05a65380-cbb3-4f32-8622-267d246018b0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:32.709363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-05a65380-cbb3-4f32-8622-267d246018b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edad36b182f9411084b55f288aa6afa1', 'request_id': 'req-05a65380-cbb3-4f32-8622-267d246018b0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:32.730645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-05a65380-cbb3-4f32-8622-267d246018b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edad36b182f9411084b55f288aa6afa1', 'request_id': 'req-05a65380-cbb3-4f32-8622-267d246018b0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:32.778749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-05a65380-cbb3-4f32-8622-267d246018b0 admin admin] The token cache is being invalidate because project edad36b182f9411084b55f288aa6afa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:32.780912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-05a65380-cbb3-4f32-8622-267d246018b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:32.784644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-05a65380-cbb3-4f32-8622-267d246018b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:32.786910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 857/1703] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:32 2022] DELETE /identity/v3/projects/edad36b182f9411084b55f288aa6afa1 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:32.789754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40308409-66d8-4253-ae1d-fabb2ef38f16 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:32.792412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40308409-66d8-4253-ae1d-fabb2ef38f16 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:32.794404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40308409-66d8-4253-ae1d-fabb2ef38f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:32.856612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6cbb4-b588-40c7-b506-1de96b2858ec None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:32.889858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-40308409-66d8-4253-ae1d-fabb2ef38f16 None None] MFA Rules not processed for user `9be671f062ed4cb793facf9ca7920263`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:32.893916 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-40308409-66d8-4253-ae1d-fabb2ef38f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:33.014650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb1825-2c9e-467d-8dcc-82417ee28397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:33.015293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb1825-2c9e-467d-8dcc-82417ee28397 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:33.015567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb1825-2c9e-467d-8dcc-82417ee28397 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:33.015832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb1825-2c9e-467d-8dcc-82417ee28397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:33.018121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-91bb1825-2c9e-467d-8dcc-82417ee28397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:33.066941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:33.125026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 858/1704] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:33.138227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40308409-66d8-4253-ae1d-fabb2ef38f16 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:33.141396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-40308409-66d8-4253-ae1d-fabb2ef38f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:33.144922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:33.214133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb1825-2c9e-467d-8dcc-82417ee28397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:33.215883 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb1825-2c9e-467d-8dcc-82417ee28397 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:33.230446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 847/1705] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:33.232389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a293-f21c-4f59-b728-202c59037220 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:33.233211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a293-f21c-4f59-b728-202c59037220 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:33.234523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a293-f21c-4f59-b728-202c59037220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:33.264745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fe87a293-f21c-4f59-b728-202c59037220 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:33.266788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fe87a293-f21c-4f59-b728-202c59037220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:33.276293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 848/1706] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:33.277814 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc13eb5-f7ee-41be-804d-d66441f5408c None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:33.278199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc13eb5-f7ee-41be-804d-d66441f5408c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:33.279118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc13eb5-f7ee-41be-804d-d66441f5408c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:33.296929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40308409-66d8-4253-ae1d-fabb2ef38f16 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:33.308889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdecc7-bc00-4960-a0a4-1e0cb17f45fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:33.310328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-dfc13eb5-f7ee-41be-804d-d66441f5408c None None] MFA Rules not processed for user `9be671f062ed4cb793facf9ca7920263`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:33.310721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdecc7-bc00-4960-a0a4-1e0cb17f45fa admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:33.311004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdecc7-bc00-4960-a0a4-1e0cb17f45fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:33.311275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdecc7-bc00-4960-a0a4-1e0cb17f45fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:33.313950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdecc7-bc00-4960-a0a4-1e0cb17f45fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:33.315356 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-dfc13eb5-f7ee-41be-804d-d66441f5408c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:33.319052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdecc7-bc00-4960-a0a4-1e0cb17f45fa admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:33.326831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 849/1707] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:33.328485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b06ab-7ada-4609-8d2c-47f7023f3fbc None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:33.329070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b06ab-7ada-4609-8d2c-47f7023f3fbc None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:33.329530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b06ab-7ada-4609-8d2c-47f7023f3fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:33.367305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-946b06ab-7ada-4609-8d2c-47f7023f3fbc None None] MFA Rules not processed for user `094abcdcd1f54a658564357056e7849a`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:33.370733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-946b06ab-7ada-4609-8d2c-47f7023f3fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:33.388759 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 859/1708] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:33 2022] POST /identity/v3/projects => generated 387 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:33.390260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765b23a-e576-45de-9155-d03494b35154 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:33.390538 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765b23a-e576-45de-9155-d03494b35154 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:33.390871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765b23a-e576-45de-9155-d03494b35154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:33.435824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5765b23a-e576-45de-9155-d03494b35154 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:33.437180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5765b23a-e576-45de-9155-d03494b35154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:33.505728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 850/1709] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:33.519670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b06ab-7ada-4609-8d2c-47f7023f3fbc None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:33.523851 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-946b06ab-7ada-4609-8d2c-47f7023f3fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:33.593322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 860/1710] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:33.594288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765b23a-e576-45de-9155-d03494b35154 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:33.620869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765b23a-e576-45de-9155-d03494b35154 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:33.625839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02b064-2828-4246-97a2-dff9b03fec14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:33.626464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02b064-2828-4246-97a2-dff9b03fec14 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:33.627183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02b064-2828-4246-97a2-dff9b03fec14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:33.627787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02b064-2828-4246-97a2-dff9b03fec14 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:33.629060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02b064-2828-4246-97a2-dff9b03fec14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:33.631804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02b064-2828-4246-97a2-dff9b03fec14 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:33.662934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 861/1711] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:33 2022] POST /identity/v3/users => generated 300 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:33.664266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02b064-2828-4246-97a2-dff9b03fec14 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:33.691295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b06ab-7ada-4609-8d2c-47f7023f3fbc None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:33.695953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02b064-2828-4246-97a2-dff9b03fec14 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:33.706121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48406f7e-21fb-439e-9318-d5c69b634093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:33.707262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48406f7e-21fb-439e-9318-d5c69b634093 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:33.708083 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48406f7e-21fb-439e-9318-d5c69b634093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:33.708381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48406f7e-21fb-439e-9318-d5c69b634093 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:33.709316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48406f7e-21fb-439e-9318-d5c69b634093 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:33.710057 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48406f7e-21fb-439e-9318-d5c69b634093 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:33.715033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48406f7e-21fb-439e-9318-d5c69b634093 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:33.741218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 862/1712] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:33.742753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48406f7e-21fb-439e-9318-d5c69b634093 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:33.754799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48406f7e-21fb-439e-9318-d5c69b634093 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:33.769569 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c7966-d46b-4abd-a50e-55bf35c65c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:33.770325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c7966-d46b-4abd-a50e-55bf35c65c75 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:33.770604 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c7966-d46b-4abd-a50e-55bf35c65c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:33.770937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c7966-d46b-4abd-a50e-55bf35c65c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:33.772433 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c7966-d46b-4abd-a50e-55bf35c65c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:33.774378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c7966-d46b-4abd-a50e-55bf35c65c75 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:33.788990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 863/1713] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:33.790111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c7966-d46b-4abd-a50e-55bf35c65c75 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:33.807909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c7966-d46b-4abd-a50e-55bf35c65c75 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:33.814667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e20625-a8e4-4bdb-af11-391ad0b645a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:33.815476 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e20625-a8e4-4bdb-af11-391ad0b645a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:33.815756 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e20625-a8e4-4bdb-af11-391ad0b645a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:33.816272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e20625-a8e4-4bdb-af11-391ad0b645a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:33.817024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e20625-a8e4-4bdb-af11-391ad0b645a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:33.820920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e20625-a8e4-4bdb-af11-391ad0b645a1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:33.869285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 864/1714] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:33 2022] POST /identity/v3/projects => generated 393 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:33.872457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e20625-a8e4-4bdb-af11-391ad0b645a1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:33.885554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:33.895608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:33.915272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103187e-13a5-486f-863d-9209c130c0a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:33.916105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103187e-13a5-486f-863d-9209c130c0a1 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:33.916448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103187e-13a5-486f-863d-9209c130c0a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:33.916995 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103187e-13a5-486f-863d-9209c130c0a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:34.005281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e20625-a8e4-4bdb-af11-391ad0b645a1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:34.011185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad48aa-1b09-4bd7-9b33-e4d0734c06bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:34.011982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad48aa-1b09-4bd7-9b33-e4d0734c06bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:34.013001 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad48aa-1b09-4bd7-9b33-e4d0734c06bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:34.013382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad48aa-1b09-4bd7-9b33-e4d0734c06bd admin admin] PATH_INFO: `/v3/projects/bcea60a6476746dfb372ee249c87fbe5/users/b31219095efa4c069ff4b8ce5f33728d/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:34.028063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103187e-13a5-486f-863d-9209c130c0a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:34.029995 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103187e-13a5-486f-863d-9209c130c0a1 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:34.037226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad48aa-1b09-4bd7-9b33-e4d0734c06bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcea60a6476746dfb372ee249c87fbe5, user_id=b31219095efa4c069ff4b8ce5f33728d, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:34.040056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad48aa-1b09-4bd7-9b33-e4d0734c06bd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:34.053615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 851/1715] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:34.054696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103187e-13a5-486f-863d-9209c130c0a1 service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:34.062086 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 865/1716] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:33 2022] PUT /identity/v3/projects/bcea60a6476746dfb372ee249c87fbe5/users/b31219095efa4c069ff4b8ce5f33728d/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:34.067341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad48aa-1b09-4bd7-9b33-e4d0734c06bd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:34.090994 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103187e-13a5-486f-863d-9209c130c0a1 service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:34.201464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad48aa-1b09-4bd7-9b33-e4d0734c06bd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:34.214248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb80a7d-122e-467b-8732-fd0ba6fc02f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:34.214895 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb80a7d-122e-467b-8732-fd0ba6fc02f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:34.215322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb80a7d-122e-467b-8732-fd0ba6fc02f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:34.215601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb80a7d-122e-467b-8732-fd0ba6fc02f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:34.216561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb80a7d-122e-467b-8732-fd0ba6fc02f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:34.217919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb80a7d-122e-467b-8732-fd0ba6fc02f8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:34.243226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa539b-a817-40fe-9aeb-ea8bc5b4e649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:34.246062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa539b-a817-40fe-9aeb-ea8bc5b4e649 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:34.246722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa539b-a817-40fe-9aeb-ea8bc5b4e649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:34.250897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 866/1717] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:28:34.251357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa539b-a817-40fe-9aeb-ea8bc5b4e649 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:34.252257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa539b-a817-40fe-9aeb-ea8bc5b4e649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:34.256743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb80a7d-122e-467b-8732-fd0ba6fc02f8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:34.257298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa539b-a817-40fe-9aeb-ea8bc5b4e649 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:34.288373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb80a7d-122e-467b-8732-fd0ba6fc02f8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:34.296747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83496564-5efa-4a3c-b77e-3f5cadc11300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:34.307039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83496564-5efa-4a3c-b77e-3f5cadc11300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:34.307039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83496564-5efa-4a3c-b77e-3f5cadc11300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:34.307039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83496564-5efa-4a3c-b77e-3f5cadc11300 admin admin] PATH_INFO: `/v3/projects/bcea60a6476746dfb372ee249c87fbe5/users/b31219095efa4c069ff4b8ce5f33728d/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:34.314775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 852/1718] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:34 2022] POST /identity/v3/users => generated 297 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:34.314775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa539b-a817-40fe-9aeb-ea8bc5b4e649 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:34.317219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-53aa539b-a817-40fe-9aeb-ea8bc5b4e649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:34.325917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83496564-5efa-4a3c-b77e-3f5cadc11300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcea60a6476746dfb372ee249c87fbe5, user_id=b31219095efa4c069ff4b8ce5f33728d, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:34.330133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83496564-5efa-4a3c-b77e-3f5cadc11300 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:34.340423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa539b-a817-40fe-9aeb-ea8bc5b4e649 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:34.371776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 867/1719] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:34 2022] PUT /identity/v3/projects/bcea60a6476746dfb372ee249c87fbe5/users/b31219095efa4c069ff4b8ce5f33728d/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:34.372980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83496564-5efa-4a3c-b77e-3f5cadc11300 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:34.537283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83496564-5efa-4a3c-b77e-3f5cadc11300 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:34.540042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:34.541287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:34.541709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:34.541958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05b96b-589c-4481-bc11-857fb62528a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:34.542320 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] PATH_INFO: `/v3/projects/ce667700f6f246f3b67ca6628c67a2c5` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:34.543157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05b96b-589c-4481-bc11-857fb62528a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:34.543157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05b96b-589c-4481-bc11-857fb62528a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:34.543446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05b96b-589c-4481-bc11-857fb62528a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:34.544211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05b96b-589c-4481-bc11-857fb62528a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:34.546279 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05b96b-589c-4481-bc11-857fb62528a2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:34.547023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce667700f6f246f3b67ca6628c67a2c5)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:34.549658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:34.566086 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 868/1720] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:34.569973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87d285-63f9-4ba4-ad18-e4e716e8433d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:34.571633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87d285-63f9-4ba4-ad18-e4e716e8433d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:34.572295 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87d285-63f9-4ba4-ad18-e4e716e8433d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:34.612664 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:34.633769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ea87d285-63f9-4ba4-ad18-e4e716e8433d None None] MFA Rules not processed for user `b31219095efa4c069ff4b8ce5f33728d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:34.635914 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ea87d285-63f9-4ba4-ad18-e4e716e8433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:34.642518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:34.662649 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce667700f6f246f3b67ca6628c67a2c5', 'request_id': 'req-34cea48f-aa84-420d-8134-ae3032521be7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:34.680776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce667700f6f246f3b67ca6628c67a2c5', 'request_id': 'req-34cea48f-aa84-420d-8134-ae3032521be7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:34.699983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce667700f6f246f3b67ca6628c67a2c5', 'request_id': 'req-34cea48f-aa84-420d-8134-ae3032521be7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:34.718553 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] The token cache is being invalidate because project ce667700f6f246f3b67ca6628c67a2c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:34.718553 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:34.719603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:34.727001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 853/1721] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:34 2022] DELETE /identity/v3/projects/ce667700f6f246f3b67ca6628c67a2c5 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:34.727856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:34.730589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:34.816996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 869/1722] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:34 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:34.818556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e28a93-dba8-4f22-b9a5-98735aaa8842 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:34.819526 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e28a93-dba8-4f22-b9a5-98735aaa8842 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:34.820062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e28a93-dba8-4f22-b9a5-98735aaa8842 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:34.822765 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 870/1723] 10.222.0.60 () {54 vars in 912 bytes} [Sat Sep 17 04:28:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:28:34.823779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e28a93-dba8-4f22-b9a5-98735aaa8842 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:34.825962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a2e28a93-dba8-4f22-b9a5-98735aaa8842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:34.832219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cea48f-aa84-420d-8134-ae3032521be7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:34.837964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0b2c8-ff5d-4339-bb92-1f5e0ae85ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:34.838763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0b2c8-ff5d-4339-bb92-1f5e0ae85ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:34.839152 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0b2c8-ff5d-4339-bb92-1f5e0ae85ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:34.839529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0b2c8-ff5d-4339-bb92-1f5e0ae85ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:34.840476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0b2c8-ff5d-4339-bb92-1f5e0ae85ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:34.842756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0b2c8-ff5d-4339-bb92-1f5e0ae85ff7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:34.857149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 854/1724] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:34.859169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130c92a-72da-4233-8e3c-ee64a3941592 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:34.859690 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130c92a-72da-4233-8e3c-ee64a3941592 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:34.860221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130c92a-72da-4233-8e3c-ee64a3941592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:34.874773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e28a93-dba8-4f22-b9a5-98735aaa8842 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:34.884847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb55627-36a3-43f5-9a18-d30e77a35ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:34.885557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb55627-36a3-43f5-9a18-d30e77a35ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:34.885834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb55627-36a3-43f5-9a18-d30e77a35ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:34.886160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb55627-36a3-43f5-9a18-d30e77a35ee1 admin admin] PATH_INFO: `/v3/users/847c984269cf4bcfbd52558e3db96373` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:34.887503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb55627-36a3-43f5-9a18-d30e77a35ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=847c984269cf4bcfbd52558e3db96373)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:34.889191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f130c92a-72da-4233-8e3c-ee64a3941592 None None] MFA Rules not processed for user `ed4e695113f843fda52ce62792a36bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:34.889559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb55627-36a3-43f5-9a18-d30e77a35ee1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:34.890877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f130c92a-72da-4233-8e3c-ee64a3941592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:34.971522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-edb55627-36a3-43f5-9a18-d30e77a35ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '847c984269cf4bcfbd52558e3db96373'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:34.975114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 855/1725] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:34.976105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130c92a-72da-4233-8e3c-ee64a3941592 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:34.978097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f130c92a-72da-4233-8e3c-ee64a3941592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:34.985134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-edb55627-36a3-43f5-9a18-d30e77a35ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '847c984269cf4bcfbd52558e3db96373'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:34.985856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-edb55627-36a3-43f5-9a18-d30e77a35ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '847c984269cf4bcfbd52558e3db96373'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:34.995304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-edb55627-36a3-43f5-9a18-d30e77a35ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '847c984269cf4bcfbd52558e3db96373'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:35.014023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-edb55627-36a3-43f5-9a18-d30e77a35ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '847c984269cf4bcfbd52558e3db96373'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:35.019799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 871/1726] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:34 2022] DELETE /identity/v3/users/847c984269cf4bcfbd52558e3db96373 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:35.021525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333add2-c38d-418a-aa35-458ca7dcdb46 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:35.022450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333add2-c38d-418a-aa35-458ca7dcdb46 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:35.022834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333add2-c38d-418a-aa35-458ca7dcdb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:35.084576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7333add2-c38d-418a-aa35-458ca7dcdb46 None None] MFA Rules not processed for user `c9360be7ee9e4739b20c59955d28da47`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:35.086181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7333add2-c38d-418a-aa35-458ca7dcdb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:35.102274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130c92a-72da-4233-8e3c-ee64a3941592 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:35.139083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:35.213827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 872/1727] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:28:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:35.214836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333add2-c38d-418a-aa35-458ca7dcdb46 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:35.216288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7333add2-c38d-418a-aa35-458ca7dcdb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:35.223750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e277e7b-aa39-4283-b19b-e95d36c209e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:35.224689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e277e7b-aa39-4283-b19b-e95d36c209e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:35.225138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e277e7b-aa39-4283-b19b-e95d36c209e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:35.225594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e277e7b-aa39-4283-b19b-e95d36c209e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:35.228842 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:35.232898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4e277e7b-aa39-4283-b19b-e95d36c209e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:35.386307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333add2-c38d-418a-aa35-458ca7dcdb46 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:35.391918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e277e7b-aa39-4283-b19b-e95d36c209e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:35.394276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e277e7b-aa39-4283-b19b-e95d36c209e0 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:35.401896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2a2ff-8560-4946-8069-a2361a22675b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:35.403573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2a2ff-8560-4946-8069-a2361a22675b admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:35.404030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2a2ff-8560-4946-8069-a2361a22675b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:35.404770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2a2ff-8560-4946-8069-a2361a22675b admin admin] PATH_INFO: `/v3/projects/b29d0d22659b47b1a14d249a49e94c9b/users/31092239f73d4bc69bb2d286af03811a/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:35.408690 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 856/1728] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:35.410424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58a3e6-6b6d-4af2-84fe-2eba44811a75 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:35.410733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58a3e6-6b6d-4af2-84fe-2eba44811a75 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:35.411082 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58a3e6-6b6d-4af2-84fe-2eba44811a75 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:35.412522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 857/1729] 10.222.0.60 () {54 vars in 912 bytes} [Sat Sep 17 04:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:28:35.413647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58a3e6-6b6d-4af2-84fe-2eba44811a75 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:35.417283 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1b58a3e6-6b6d-4af2-84fe-2eba44811a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:35.449118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58a3e6-6b6d-4af2-84fe-2eba44811a75 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:35.457259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b40a72-505d-4a52-b2ba-edd4ad3d2bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:35.457898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b40a72-505d-4a52-b2ba-edd4ad3d2bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:35.458176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b40a72-505d-4a52-b2ba-edd4ad3d2bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:35.458447 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b40a72-505d-4a52-b2ba-edd4ad3d2bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:35.475524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2a2ff-8560-4946-8069-a2361a22675b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29d0d22659b47b1a14d249a49e94c9b, user_id=31092239f73d4bc69bb2d286af03811a, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:35.478970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2a2ff-8560-4946-8069-a2361a22675b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:35.485167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b40a72-505d-4a52-b2ba-edd4ad3d2bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:35.487166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b40a72-505d-4a52-b2ba-edd4ad3d2bc2 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:35.498727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 858/1730] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:35.500359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabaf1e-5ef6-47eb-982a-ab603c5bace8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:35.500969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabaf1e-5ef6-47eb-982a-ab603c5bace8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:35.501449 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabaf1e-5ef6-47eb-982a-ab603c5bace8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:35.537641 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 873/1731] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:35 2022] PUT /identity/v3/projects/b29d0d22659b47b1a14d249a49e94c9b/users/31092239f73d4bc69bb2d286af03811a/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:35.537641 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2a2ff-8560-4946-8069-a2361a22675b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:35.548180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f6d2a2ff-8560-4946-8069-a2361a22675b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:35.564174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-baabaf1e-5ef6-47eb-982a-ab603c5bace8 None None] MFA Rules not processed for user `c9360be7ee9e4739b20c59955d28da47`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:35.572537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-baabaf1e-5ef6-47eb-982a-ab603c5bace8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:35.723557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2a2ff-8560-4946-8069-a2361a22675b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:35.729182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 859/1732] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:28:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:35.731220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c44fcd-0bed-4b32-80db-4e5d458488a1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:35.731609 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c44fcd-0bed-4b32-80db-4e5d458488a1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:35.732733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c44fcd-0bed-4b32-80db-4e5d458488a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:35.749572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:35.750479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:35.750990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:35.751840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] PATH_INFO: `/v3/projects/d2969ddbda9b45bf8994696ceba2b564` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:35.760205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2969ddbda9b45bf8994696ceba2b564)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:35.766940 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-90c44fcd-0bed-4b32-80db-4e5d458488a1 None None] MFA Rules not processed for user `31092239f73d4bc69bb2d286af03811a`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:35.769514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-90c44fcd-0bed-4b32-80db-4e5d458488a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:35.772727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:35.892170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:35.895089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2969ddbda9b45bf8994696ceba2b564', 'request_id': 'req-063bad2c-fde8-4b56-a173-d08a9ae95a40'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:35.909436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2969ddbda9b45bf8994696ceba2b564', 'request_id': 'req-063bad2c-fde8-4b56-a173-d08a9ae95a40'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:35.911204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 860/1733] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:35.912715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c44fcd-0bed-4b32-80db-4e5d458488a1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:35.915438 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-90c44fcd-0bed-4b32-80db-4e5d458488a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:35.918106 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2969ddbda9b45bf8994696ceba2b564', 'request_id': 'req-063bad2c-fde8-4b56-a173-d08a9ae95a40'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:35.933673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c44fcd-0bed-4b32-80db-4e5d458488a1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:35.945368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] The token cache is being invalidate because project d2969ddbda9b45bf8994696ceba2b564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:35.945858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:35.946874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:35.950636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 874/1734] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:35 2022] DELETE /identity/v3/projects/d2969ddbda9b45bf8994696ceba2b564 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:35.952248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:35.955183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:35.984561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:36.016066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa1a76-5293-4db3-959f-92540656a89e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:36.016826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa1a76-5293-4db3-959f-92540656a89e service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:36.017334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa1a76-5293-4db3-959f-92540656a89e service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:36.017605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa1a76-5293-4db3-959f-92540656a89e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:36.019669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-36fa1a76-5293-4db3-959f-92540656a89e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:36.031595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa1a76-5293-4db3-959f-92540656a89e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:36.033490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa1a76-5293-4db3-959f-92540656a89e service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:36.044136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 861/1735] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:36.044779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa1a76-5293-4db3-959f-92540656a89e service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:36.047103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-36fa1a76-5293-4db3-959f-92540656a89e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:36.082144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bad2c-fde8-4b56-a173-d08a9ae95a40 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:36.088437 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:36.089968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:36.089968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:36.089968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] PATH_INFO: `/v3/users/16237c4c3d574a5393476f7e078c652c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:36.091021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16237c4c3d574a5393476f7e078c652c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:36.093305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:36.165455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa1a76-5293-4db3-959f-92540656a89e service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:36.176425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81712f50-c3fd-423b-b232-0d4cfb4406fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:36.177161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81712f50-c3fd-423b-b232-0d4cfb4406fd admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:36.177366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81712f50-c3fd-423b-b232-0d4cfb4406fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:36.177645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81712f50-c3fd-423b-b232-0d4cfb4406fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:36.179061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81712f50-c3fd-423b-b232-0d4cfb4406fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:36.182665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81712f50-c3fd-423b-b232-0d4cfb4406fd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:36.227904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16237c4c3d574a5393476f7e078c652c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:36.235258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 862/1736] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:36 2022] POST /identity/v3/projects => generated 391 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:36.236822 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079793b-4e8d-451d-a909-9760a0456569 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:36.237757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079793b-4e8d-451d-a909-9760a0456569 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:36.238199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079793b-4e8d-451d-a909-9760a0456569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:36.243446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16237c4c3d574a5393476f7e078c652c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:36.245583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16237c4c3d574a5393476f7e078c652c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:36.255854 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16237c4c3d574a5393476f7e078c652c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:36.273594 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16237c4c3d574a5393476f7e078c652c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:36.281293 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 875/1737] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:35 2022] DELETE /identity/v3/users/16237c4c3d574a5393476f7e078c652c => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:36.282649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:36.284495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:36.291567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-8079793b-4e8d-451d-a909-9760a0456569 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:36.293533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8079793b-4e8d-451d-a909-9760a0456569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:36.365912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eceb6f-111d-4ff5-9a96-da4d016d7760 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:36.375818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f22220-10cc-4c44-afe5-38c2dfee27e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:36.376463 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22220-10cc-4c44-afe5-38c2dfee27e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:36.376744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22220-10cc-4c44-afe5-38c2dfee27e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:36.377015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22220-10cc-4c44-afe5-38c2dfee27e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:36.377689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f22220-10cc-4c44-afe5-38c2dfee27e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:36.383187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f22220-10cc-4c44-afe5-38c2dfee27e9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:36.422789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 863/1738] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:36.424384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60374aa-0b70-417d-8a8b-b71fccdfb909 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:36.424667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60374aa-0b70-417d-8a8b-b71fccdfb909 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:36.424936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60374aa-0b70-417d-8a8b-b71fccdfb909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:36.434772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 876/1739] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:36 2022] POST /identity/v3/users => generated 302 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:36.437631 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bd4f4-5ad9-4264-b5a4-a451ff92d8a5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:36.437957 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bd4f4-5ad9-4264-b5a4-a451ff92d8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:36.438669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bd4f4-5ad9-4264-b5a4-a451ff92d8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:36.447053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e60374aa-0b70-417d-8a8b-b71fccdfb909 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:36.448313 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e60374aa-0b70-417d-8a8b-b71fccdfb909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:36.457525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 864/1740] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:36.458529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60374aa-0b70-417d-8a8b-b71fccdfb909 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:36.478340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60374aa-0b70-417d-8a8b-b71fccdfb909 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:36.483614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183f6c4-1c38-4ea8-8b56-e07dee7aabc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:36.483885 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-749bd4f4-5ad9-4264-b5a4-a451ff92d8a5 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:36.484762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183f6c4-1c38-4ea8-8b56-e07dee7aabc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:36.485176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-749bd4f4-5ad9-4264-b5a4-a451ff92d8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:36.485558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183f6c4-1c38-4ea8-8b56-e07dee7aabc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:36.485971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183f6c4-1c38-4ea8-8b56-e07dee7aabc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:36.486917 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183f6c4-1c38-4ea8-8b56-e07dee7aabc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:36.488889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183f6c4-1c38-4ea8-8b56-e07dee7aabc6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:36.502491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 865/1741] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:36.503456 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183f6c4-1c38-4ea8-8b56-e07dee7aabc6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:36.505760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3183f6c4-1c38-4ea8-8b56-e07dee7aabc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:36.512649 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:36.540801 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 877/1742] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:36.542525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bd4f4-5ad9-4264-b5a4-a451ff92d8a5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:36.579450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bd4f4-5ad9-4264-b5a4-a451ff92d8a5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:36.594016 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828561fa-3906-4983-bcfa-326fac588a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:36.595883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828561fa-3906-4983-bcfa-326fac588a78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:36.598834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828561fa-3906-4983-bcfa-326fac588a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:36.599178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828561fa-3906-4983-bcfa-326fac588a78 admin admin] PATH_INFO: `/v3/projects/3d8d7d6a67644cb19dd5318b39ebc85e/users/c6b814ffce8c41268e260ec60e81f81d/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:36.612434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183f6c4-1c38-4ea8-8b56-e07dee7aabc6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:36.616877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a30e2-a819-43d2-9588-167962c43838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:36.617923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a30e2-a819-43d2-9588-167962c43838 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:36.618302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a30e2-a819-43d2-9588-167962c43838 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:36.618702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a30e2-a819-43d2-9588-167962c43838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:36.639018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a30e2-a819-43d2-9588-167962c43838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:36.646255 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a30e2-a819-43d2-9588-167962c43838 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:36.662618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828561fa-3906-4983-bcfa-326fac588a78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d8d7d6a67644cb19dd5318b39ebc85e, user_id=c6b814ffce8c41268e260ec60e81f81d, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:36.665870 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828561fa-3906-4983-bcfa-326fac588a78 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:36.668620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:36.697531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 866/1743] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:36.698939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a30e2-a819-43d2-9588-167962c43838 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:36.713942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a30e2-a819-43d2-9588-167962c43838 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:36.722377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 878/1744] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:36 2022] PUT /identity/v3/projects/3d8d7d6a67644cb19dd5318b39ebc85e/users/c6b814ffce8c41268e260ec60e81f81d/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:36.723602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828561fa-3906-4983-bcfa-326fac588a78 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:36.738298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-828561fa-3906-4983-bcfa-326fac588a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:36.847590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa6e3b-a26f-44a7-b850-b5402457fa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:36.848251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa6e3b-a26f-44a7-b850-b5402457fa0e admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:36.848537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa6e3b-a26f-44a7-b850-b5402457fa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:36.848799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa6e3b-a26f-44a7-b850-b5402457fa0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:36.849512 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa6e3b-a26f-44a7-b850-b5402457fa0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:36.857796 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa6e3b-a26f-44a7-b850-b5402457fa0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:36.860184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa6e3b-a26f-44a7-b850-b5402457fa0e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:36.912945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 867/1745] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:36.914531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa6e3b-a26f-44a7-b850-b5402457fa0e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:36.950511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa6e3b-a26f-44a7-b850-b5402457fa0e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:36.973905 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccca1f-46fa-4451-8fb8-d2119cbd03b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:36.974973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccca1f-46fa-4451-8fb8-d2119cbd03b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:36.975467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccca1f-46fa-4451-8fb8-d2119cbd03b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:36.975946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccca1f-46fa-4451-8fb8-d2119cbd03b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:36.976806 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccca1f-46fa-4451-8fb8-d2119cbd03b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:36.979057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccca1f-46fa-4451-8fb8-d2119cbd03b0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:36.982694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828561fa-3906-4983-bcfa-326fac588a78 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:36.998138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 868/1746] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:36.999536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dacf0-f7dd-471f-b6b2-bd28996e6434 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:37.000180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dacf0-f7dd-471f-b6b2-bd28996e6434 service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:37.000452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dacf0-f7dd-471f-b6b2-bd28996e6434 service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:37.000827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dacf0-f7dd-471f-b6b2-bd28996e6434 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:37.002991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccca1f-46fa-4451-8fb8-d2119cbd03b0 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:37.005874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3e0dacf0-f7dd-471f-b6b2-bd28996e6434 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:37.014010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-47ccca1f-46fa-4451-8fb8-d2119cbd03b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:37.044678 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dacf0-f7dd-471f-b6b2-bd28996e6434 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:37.057852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dacf0-f7dd-471f-b6b2-bd28996e6434 service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:37.061177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccca1f-46fa-4451-8fb8-d2119cbd03b0 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:37.074730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:37.075560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:37.075941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:37.076319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] PATH_INFO: `/v3/users/3546689f8f93481c89322ff9fe4dbcec` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:37.080557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3546689f8f93481c89322ff9fe4dbcec)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:37.082705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:37.102138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 879/1747] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:37.103374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dacf0-f7dd-471f-b6b2-bd28996e6434 service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:37.141044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dacf0-f7dd-471f-b6b2-bd28996e6434 service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:37.162945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb0c15-15f7-41dc-b962-ac1e959ac3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:37.163498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb0c15-15f7-41dc-b962-ac1e959ac3d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:37.163675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb0c15-15f7-41dc-b962-ac1e959ac3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:37.163931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb0c15-15f7-41dc-b962-ac1e959ac3d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:37.165349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb0c15-15f7-41dc-b962-ac1e959ac3d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:37.170769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb0c15-15f7-41dc-b962-ac1e959ac3d7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:37.207043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3546689f8f93481c89322ff9fe4dbcec'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:37.215378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 880/1748] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:37 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:37.217522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cf6e6-38b8-4d71-a040-3bea558e911b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:37.217862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cf6e6-38b8-4d71-a040-3bea558e911b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:37.218183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cf6e6-38b8-4d71-a040-3bea558e911b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:37.221556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 881/1749] 10.222.0.60 () {54 vars in 961 bytes} [Sat Sep 17 04:28:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:28:37.222472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cf6e6-38b8-4d71-a040-3bea558e911b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:37.228552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3546689f8f93481c89322ff9fe4dbcec'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:37.229229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3546689f8f93481c89322ff9fe4dbcec'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:37.243787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3546689f8f93481c89322ff9fe4dbcec'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:37.261224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3546689f8f93481c89322ff9fe4dbcec'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:37.269896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 869/1750] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:37 2022] DELETE /identity/v3/users/3546689f8f93481c89322ff9fe4dbcec => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:37.270934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:37.273190 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:37.280388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:37.399145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cf6e6-38b8-4d71-a040-3bea558e911b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:37.449228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8b929-8cf3-413c-a1a1-92fa40f5d770 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:37.465918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe8412-8936-440a-8562-db0707250db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:37.466625 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe8412-8936-440a-8562-db0707250db9 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:37.466841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe8412-8936-440a-8562-db0707250db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:37.467163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe8412-8936-440a-8562-db0707250db9 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:37.467807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe8412-8936-440a-8562-db0707250db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:37.469838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe8412-8936-440a-8562-db0707250db9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:37.525793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 870/1751] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:37 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:37.527222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419a96b-ac68-4678-8c71-ff3f078c92f4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:37.527516 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419a96b-ac68-4678-8c71-ff3f078c92f4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:37.528084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419a96b-ac68-4678-8c71-ff3f078c92f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:37.570809 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c419a96b-ac68-4678-8c71-ff3f078c92f4 None None] MFA Rules not processed for user `e6d3b7887c13466ab8e3d9737f907c56`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:37.574316 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c419a96b-ac68-4678-8c71-ff3f078c92f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:37.575090 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce6e3a-e35d-41c4-b14c-02666d227a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:37.575860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce6e3a-e35d-41c4-b14c-02666d227a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:37.576572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce6e3a-e35d-41c4-b14c-02666d227a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:37.578285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce6e3a-e35d-41c4-b14c-02666d227a33 admin admin] PATH_INFO: `/v3/projects/3d8d7d6a67644cb19dd5318b39ebc85e/users/c6b814ffce8c41268e260ec60e81f81d/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:37.583457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce6e3a-e35d-41c4-b14c-02666d227a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d8d7d6a67644cb19dd5318b39ebc85e, user_id=c6b814ffce8c41268e260ec60e81f81d, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:37.593119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce6e3a-e35d-41c4-b14c-02666d227a33 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:37.649990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 882/1752] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:37 2022] PUT /identity/v3/projects/3d8d7d6a67644cb19dd5318b39ebc85e/users/c6b814ffce8c41268e260ec60e81f81d/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:37.651659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e203800-dfb3-4123-898a-3183d89d1bba None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:37.651945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e203800-dfb3-4123-898a-3183d89d1bba None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:37.657844 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e203800-dfb3-4123-898a-3183d89d1bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:37.684658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:37.711739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4e203800-dfb3-4123-898a-3183d89d1bba None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:37.714497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4e203800-dfb3-4123-898a-3183d89d1bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:37.738971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 871/1753] 10.222.0.60 () {58 vars in 1058 bytes} [Sat Sep 17 04:28:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:37.743089 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419a96b-ac68-4678-8c71-ff3f078c92f4 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:37.773026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419a96b-ac68-4678-8c71-ff3f078c92f4 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:37.903113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 883/1754] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:37.907931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7a703-8043-4a3b-9f40-b2ab80423f93 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:37.914721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7a703-8043-4a3b-9f40-b2ab80423f93 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:37.915114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7a703-8043-4a3b-9f40-b2ab80423f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:37.944361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed47ea-7d08-4899-8ea4-4ebd2099bfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:37.945595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed47ea-7d08-4899-8ea4-4ebd2099bfaa admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:37.945945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed47ea-7d08-4899-8ea4-4ebd2099bfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:37.946291 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed47ea-7d08-4899-8ea4-4ebd2099bfaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:37.947056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed47ea-7d08-4899-8ea4-4ebd2099bfaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:37.949396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed47ea-7d08-4899-8ea4-4ebd2099bfaa admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:37.965731 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 872/1755] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:37.966892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed47ea-7d08-4899-8ea4-4ebd2099bfaa admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:37.986045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a3f7a703-8043-4a3b-9f40-b2ab80423f93 None None] MFA Rules not processed for user `c6b814ffce8c41268e260ec60e81f81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:37.987270 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a3f7a703-8043-4a3b-9f40-b2ab80423f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:38.047745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:38.153756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed47ea-7d08-4899-8ea4-4ebd2099bfaa admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:38.172088 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e796d-be3a-4cf1-95fa-0ec753e83b15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:38.175761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e796d-be3a-4cf1-95fa-0ec753e83b15 service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:38.176078 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e796d-be3a-4cf1-95fa-0ec753e83b15 service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:38.176357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e796d-be3a-4cf1-95fa-0ec753e83b15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:38.180281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c80e796d-be3a-4cf1-95fa-0ec753e83b15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:38.202832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 884/1756] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:37 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:38.204137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7a703-8043-4a3b-9f40-b2ab80423f93 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:38.207494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a3f7a703-8043-4a3b-9f40-b2ab80423f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:38.237985 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7a703-8043-4a3b-9f40-b2ab80423f93 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:38.238994 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e796d-be3a-4cf1-95fa-0ec753e83b15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:38.240698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e796d-be3a-4cf1-95fa-0ec753e83b15 service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:38.251283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:38.252175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:38.252463 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:38.252923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] PATH_INFO: `/v3/projects/e6d64dd538004d0a88095f0b5b0f554d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:38.255099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d64dd538004d0a88095f0b5b0f554d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:38.270899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:38.284786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 873/1757] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:38.286105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e796d-be3a-4cf1-95fa-0ec753e83b15 service placement] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:38.292885 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c80e796d-be3a-4cf1-95fa-0ec753e83b15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:38.414799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d64dd538004d0a88095f0b5b0f554d', 'request_id': 'req-bd38fa76-0cd7-4653-bade-f2b13179e6fe'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:38.433929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d64dd538004d0a88095f0b5b0f554d', 'request_id': 'req-bd38fa76-0cd7-4653-bade-f2b13179e6fe'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:38.443629 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d64dd538004d0a88095f0b5b0f554d', 'request_id': 'req-bd38fa76-0cd7-4653-bade-f2b13179e6fe'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:38.510109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e796d-be3a-4cf1-95fa-0ec753e83b15 service placement] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:38.515525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] The token cache is being invalidate because project e6d64dd538004d0a88095f0b5b0f554d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:38.518209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:38.523375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:38.525947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 885/1758] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:38 2022] DELETE /identity/v3/projects/e6d64dd538004d0a88095f0b5b0f554d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:38.528935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:38.532590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:38.574238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:38.687255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38fa76-0cd7-4653-bade-f2b13179e6fe admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:38.691930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b54c8-07a7-4bc5-8af2-b20b58f2c11b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:38.692579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b54c8-07a7-4bc5-8af2-b20b58f2c11b service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:38.698463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b54c8-07a7-4bc5-8af2-b20b58f2c11b service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:38.698463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b54c8-07a7-4bc5-8af2-b20b58f2c11b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:38.698787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2987f-1c65-4e18-8580-893548d08e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:38.699284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2987f-1c65-4e18-8580-893548d08e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:38.699562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2987f-1c65-4e18-8580-893548d08e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:38.699832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2987f-1c65-4e18-8580-893548d08e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:38.700487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2987f-1c65-4e18-8580-893548d08e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:38.702933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2987f-1c65-4e18-8580-893548d08e6e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:38.705043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2a5b54c8-07a7-4bc5-8af2-b20b58f2c11b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:38.748115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 886/1759] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:38.749446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2987f-1c65-4e18-8580-893548d08e6e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:38.751402 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9bd2987f-1c65-4e18-8580-893548d08e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:38.761907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b54c8-07a7-4bc5-8af2-b20b58f2c11b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:38.765523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b54c8-07a7-4bc5-8af2-b20b58f2c11b service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:38.815962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 874/1760] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:38.816965 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b54c8-07a7-4bc5-8af2-b20b58f2c11b service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:38.819562 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2a5b54c8-07a7-4bc5-8af2-b20b58f2c11b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:38.881778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b54c8-07a7-4bc5-8af2-b20b58f2c11b service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:38.906724 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:38.907645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:38.907785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:38.908067 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] PATH_INFO: `/v3/users/b31219095efa4c069ff4b8ce5f33728d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:38.915738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b31219095efa4c069ff4b8ce5f33728d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:38.926264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:38.960480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2987f-1c65-4e18-8580-893548d08e6e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:38.968185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50b5d9-d62d-4d76-b6e3-cabfe75c85ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:38.970679 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50b5d9-d62d-4d76-b6e3-cabfe75c85ee service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:38.970784 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50b5d9-d62d-4d76-b6e3-cabfe75c85ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:38.971028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50b5d9-d62d-4d76-b6e3-cabfe75c85ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:38.976021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0d50b5d9-d62d-4d76-b6e3-cabfe75c85ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:39.085022 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b31219095efa4c069ff4b8ce5f33728d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:39.130435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b31219095efa4c069ff4b8ce5f33728d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:39.131609 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b31219095efa4c069ff4b8ce5f33728d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:39.142575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b31219095efa4c069ff4b8ce5f33728d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:39.148570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:39.164444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b31219095efa4c069ff4b8ce5f33728d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:39.185781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 875/1761] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:38 2022] DELETE /identity/v3/users/b31219095efa4c069ff4b8ce5f33728d => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:39.189238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:39.200688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50b5d9-d62d-4d76-b6e3-cabfe75c85ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:39.202493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50b5d9-d62d-4d76-b6e3-cabfe75c85ee service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:39.206633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:39.229829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 887/1762] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:38 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:39.231339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50b5d9-d62d-4d76-b6e3-cabfe75c85ee service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:39.233483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0d50b5d9-d62d-4d76-b6e3-cabfe75c85ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:39.263735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50b5d9-d62d-4d76-b6e3-cabfe75c85ee service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:39.373431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa2ce8-95c8-48f1-9337-0d1df2c569a6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:39.378689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075531d-85ca-4183-881c-f0023f1c169c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:39.379385 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075531d-85ca-4183-881c-f0023f1c169c admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:39.379665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075531d-85ca-4183-881c-f0023f1c169c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:39.379940 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075531d-85ca-4183-881c-f0023f1c169c admin admin] PATH_INFO: `/v3/projects/646448a4902f4565a093694056ddbd2d/users/800825703b0240a08e17e42e278ec607/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:39.391215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:39.402100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075531d-85ca-4183-881c-f0023f1c169c admin admin] RBAC: Authorizing `identity:create_grant(project_id=646448a4902f4565a093694056ddbd2d, user_id=800825703b0240a08e17e42e278ec607, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:39.406376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075531d-85ca-4183-881c-f0023f1c169c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:39.409057 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4011b7-5010-4805-b64d-c0c33529385d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:39.412405 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4011b7-5010-4805-b64d-c0c33529385d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:39.414972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4011b7-5010-4805-b64d-c0c33529385d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:39.415284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4011b7-5010-4805-b64d-c0c33529385d admin admin] PATH_INFO: `/v3/users/6eb1a37b6cc0412b987ee04603cba5e1` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:39.419211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4011b7-5010-4805-b64d-c0c33529385d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eb1a37b6cc0412b987ee04603cba5e1)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:39.424516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4011b7-5010-4805-b64d-c0c33529385d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:39.439249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 876/1763] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:39 2022] PUT /identity/v3/projects/646448a4902f4565a093694056ddbd2d/users/800825703b0240a08e17e42e278ec607/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:39.440698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075531d-85ca-4183-881c-f0023f1c169c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:39.450863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0075531d-85ca-4183-881c-f0023f1c169c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:39.543362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075531d-85ca-4183-881c-f0023f1c169c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:39.557054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc712e-82e1-4435-ac7b-a9e4ed5a6852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:39.558355 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc712e-82e1-4435-ac7b-a9e4ed5a6852 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:39.560112 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc712e-82e1-4435-ac7b-a9e4ed5a6852 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:39.560572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc712e-82e1-4435-ac7b-a9e4ed5a6852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:39.567470 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-68fc712e-82e1-4435-ac7b-a9e4ed5a6852 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:39.632174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d4011b7-5010-4805-b64d-c0c33529385d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eb1a37b6cc0412b987ee04603cba5e1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:39.649534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d4011b7-5010-4805-b64d-c0c33529385d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eb1a37b6cc0412b987ee04603cba5e1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:39.650371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d4011b7-5010-4805-b64d-c0c33529385d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eb1a37b6cc0412b987ee04603cba5e1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:39.687391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d4011b7-5010-4805-b64d-c0c33529385d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eb1a37b6cc0412b987ee04603cba5e1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:39.710530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d4011b7-5010-4805-b64d-c0c33529385d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eb1a37b6cc0412b987ee04603cba5e1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:39.727791 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 888/1764] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:39 2022] DELETE /identity/v3/users/6eb1a37b6cc0412b987ee04603cba5e1 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:39.729923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa7257-a37a-43f4-a9d7-c3cdf9b730d8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:39.730299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa7257-a37a-43f4-a9d7-c3cdf9b730d8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:39.730981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa7257-a37a-43f4-a9d7-c3cdf9b730d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:39.750499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc712e-82e1-4435-ac7b-a9e4ed5a6852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:39.752550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc712e-82e1-4435-ac7b-a9e4ed5a6852 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:39.767320 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 877/1765] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:39.768447 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc712e-82e1-4435-ac7b-a9e4ed5a6852 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:39.769982 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-68fc712e-82e1-4435-ac7b-a9e4ed5a6852 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:39.828346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-74aa7257-a37a-43f4-a9d7-c3cdf9b730d8 None None] MFA Rules not processed for user `800825703b0240a08e17e42e278ec607`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:39.834734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-74aa7257-a37a-43f4-a9d7-c3cdf9b730d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:39.849261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:39.972309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc712e-82e1-4435-ac7b-a9e4ed5a6852 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:40.072205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 889/1766] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:40.073879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62533f60-a697-4ae0-867b-85a44c7b3446 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:40.074169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62533f60-a697-4ae0-867b-85a44c7b3446 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:40.079914 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62533f60-a697-4ae0-867b-85a44c7b3446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:40.123413 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:40.124435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:40.124708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:40.124987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] PATH_INFO: `/v3/projects/f9a4026cd378405d9320d9d4dae775ac` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:40.127456 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a4026cd378405d9320d9d4dae775ac)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:40.129855 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:40.133785 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-62533f60-a697-4ae0-867b-85a44c7b3446 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:40.137028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-62533f60-a697-4ae0-867b-85a44c7b3446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:40.171562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 890/1767] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:40.172663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62533f60-a697-4ae0-867b-85a44c7b3446 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:40.186646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-62533f60-a697-4ae0-867b-85a44c7b3446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:40.231715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a4026cd378405d9320d9d4dae775ac', 'request_id': 'req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:40.235647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62533f60-a697-4ae0-867b-85a44c7b3446 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:40.244631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a4026cd378405d9320d9d4dae775ac', 'request_id': 'req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:40.255422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a4026cd378405d9320d9d4dae775ac', 'request_id': 'req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:40.271913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] The token cache is being invalidate because project f9a4026cd378405d9320d9d4dae775ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:40.272229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:40.273328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:40.275621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 878/1768] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:39 2022] DELETE /identity/v3/projects/f9a4026cd378405d9320d9d4dae775ac => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:40.276656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:40.279342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:40.378982 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8220ef1-4095-4ff9-8a7c-1143edd17b9c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:40.388490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffe2cb-8d2c-49e3-b019-e4dbbb8c72f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:40.393870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffe2cb-8d2c-49e3-b019-e4dbbb8c72f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:40.394504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffe2cb-8d2c-49e3-b019-e4dbbb8c72f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:40.395906 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffe2cb-8d2c-49e3-b019-e4dbbb8c72f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:40.405162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7fffe2cb-8d2c-49e3-b019-e4dbbb8c72f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:40.466271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:40.476537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de4e56-5208-4926-bb04-a8512e0d2e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:40.478601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de4e56-5208-4926-bb04-a8512e0d2e73 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:40.478601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de4e56-5208-4926-bb04-a8512e0d2e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:40.478601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de4e56-5208-4926-bb04-a8512e0d2e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:40.481658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de4e56-5208-4926-bb04-a8512e0d2e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:40.487851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de4e56-5208-4926-bb04-a8512e0d2e73 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:40.499674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:40.584074 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 891/1769] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:40 2022] POST /identity/v3/projects => generated 375 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:40.585369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de4e56-5208-4926-bb04-a8512e0d2e73 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:40.590281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-11de4e56-5208-4926-bb04-a8512e0d2e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:40.673520 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffe2cb-8d2c-49e3-b019-e4dbbb8c72f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:40.677879 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffe2cb-8d2c-49e3-b019-e4dbbb8c72f7 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:40.690202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 879/1770] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:40.691775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd8701-95fa-4228-b5de-827cfbe3ca59 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:40.692160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd8701-95fa-4228-b5de-827cfbe3ca59 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:40.692500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd8701-95fa-4228-b5de-827cfbe3ca59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:40.719827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-37fd8701-95fa-4228-b5de-827cfbe3ca59 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:40.721587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-37fd8701-95fa-4228-b5de-827cfbe3ca59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:40.790326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de4e56-5208-4926-bb04-a8512e0d2e73 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:40.803321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:40.804004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:40.804285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:40.804560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] PATH_INFO: `/v3/projects/bcea60a6476746dfb372ee249c87fbe5` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:40.813678 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcea60a6476746dfb372ee249c87fbe5)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:40.815627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:40.830059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 880/1771] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:40.857977 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd8701-95fa-4228-b5de-827cfbe3ca59 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:40.863135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-37fd8701-95fa-4228-b5de-827cfbe3ca59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:40.914790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd8701-95fa-4228-b5de-827cfbe3ca59 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:40.927838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680120b-7835-4b9c-9569-e926cece5f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:40.930039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680120b-7835-4b9c-9569-e926cece5f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:40.930676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680120b-7835-4b9c-9569-e926cece5f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:40.931777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680120b-7835-4b9c-9569-e926cece5f56 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:40.936231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4680120b-7835-4b9c-9569-e926cece5f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:40.941618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4680120b-7835-4b9c-9569-e926cece5f56 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:40.953232 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcea60a6476746dfb372ee249c87fbe5', 'request_id': 'req-01dd36c4-84d6-4069-956d-a470ea3a76d2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:40.972648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcea60a6476746dfb372ee249c87fbe5', 'request_id': 'req-01dd36c4-84d6-4069-956d-a470ea3a76d2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:40.979931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcea60a6476746dfb372ee249c87fbe5', 'request_id': 'req-01dd36c4-84d6-4069-956d-a470ea3a76d2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:40.986195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 881/1772] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:40 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:40.987912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920198-4489-43b2-8d30-f866cd1810c8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:40.988506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920198-4489-43b2-8d30-f866cd1810c8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:40.988955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920198-4489-43b2-8d30-f866cd1810c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:41.038546 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] The token cache is being invalidate because project bcea60a6476746dfb372ee249c87fbe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:41.039024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:41.042421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:41.047164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 892/1773] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:40 2022] DELETE /identity/v3/projects/bcea60a6476746dfb372ee249c87fbe5 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:41.051262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:41.053163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a4920198-4489-43b2-8d30-f866cd1810c8 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:41.054654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a4920198-4489-43b2-8d30-f866cd1810c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:41.055953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:41.190729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd36c4-84d6-4069-956d-a470ea3a76d2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:41.196306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbfd598-7113-4008-a83b-c228d20ce592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:41.196949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfd598-7113-4008-a83b-c228d20ce592 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:41.197264 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfd598-7113-4008-a83b-c228d20ce592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:41.197587 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfd598-7113-4008-a83b-c228d20ce592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:41.198250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbfd598-7113-4008-a83b-c228d20ce592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:41.200894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbfd598-7113-4008-a83b-c228d20ce592 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:41.206116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 882/1774] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:41.209243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94169a-c880-4d47-ae71-062c96893981 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:41.209536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94169a-c880-4d47-ae71-062c96893981 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:41.209986 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94169a-c880-4d47-ae71-062c96893981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:41.241715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 893/1775] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:41.243698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe44201-cd7c-4143-bed8-77fc618a3262 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:41.243984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe44201-cd7c-4143-bed8-77fc618a3262 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:41.244257 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe44201-cd7c-4143-bed8-77fc618a3262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:41.256106 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-de94169a-c880-4d47-ae71-062c96893981 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:41.257373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-de94169a-c880-4d47-ae71-062c96893981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:41.272186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 883/1776] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:41.272186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94169a-c880-4d47-ae71-062c96893981 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:41.277107 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:41.284138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94169a-c880-4d47-ae71-062c96893981 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:41.290500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6835d-de91-48b5-9f06-74651c72125e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:41.291397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6835d-de91-48b5-9f06-74651c72125e admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:41.291817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6835d-de91-48b5-9f06-74651c72125e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:41.292398 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6835d-de91-48b5-9f06-74651c72125e admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:41.293758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6835d-de91-48b5-9f06-74651c72125e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:41.294237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6835d-de91-48b5-9f06-74651c72125e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:41.298535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6835d-de91-48b5-9f06-74651c72125e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:41.316406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 884/1777] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:41.317546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6835d-de91-48b5-9f06-74651c72125e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:41.331582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6835d-de91-48b5-9f06-74651c72125e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:41.337698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb6ce8-2a61-4643-89be-7e6deed2e016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:41.339152 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb6ce8-2a61-4643-89be-7e6deed2e016 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:41.340104 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb6ce8-2a61-4643-89be-7e6deed2e016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:41.340521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb6ce8-2a61-4643-89be-7e6deed2e016 admin admin] PATH_INFO: `/v3/projects/b1a0a97dc69a468b95302dcd036766c6/users/14c5514eed194029aebe78922ece2e63/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:41.340775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1fe44201-cd7c-4143-bed8-77fc618a3262 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:41.342413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1fe44201-cd7c-4143-bed8-77fc618a3262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:41.359631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:41.368840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb6ce8-2a61-4643-89be-7e6deed2e016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a0a97dc69a468b95302dcd036766c6, user_id=14c5514eed194029aebe78922ece2e63, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:41.375511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb6ce8-2a61-4643-89be-7e6deed2e016 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:41.411397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 885/1778] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:41 2022] PUT /identity/v3/projects/b1a0a97dc69a468b95302dcd036766c6/users/14c5514eed194029aebe78922ece2e63/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:41.413467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811bf35-9d58-418e-aa70-d044a0e1b98d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:41.414083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811bf35-9d58-418e-aa70-d044a0e1b98d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:41.414221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811bf35-9d58-418e-aa70-d044a0e1b98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:41.444318 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d811bf35-9d58-418e-aa70-d044a0e1b98d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:41.449697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d811bf35-9d58-418e-aa70-d044a0e1b98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:41.456921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 894/1779] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:28:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:41.458372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe44201-cd7c-4143-bed8-77fc618a3262 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:41.472729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe44201-cd7c-4143-bed8-77fc618a3262 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:41.602623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 886/1780] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:41.607354 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811bf35-9d58-418e-aa70-d044a0e1b98d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:41.640307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811bf35-9d58-418e-aa70-d044a0e1b98d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:41.644516 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c7923-4580-455c-a3f1-50377a99b885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:41.646072 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c7923-4580-455c-a3f1-50377a99b885 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:41.646385 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c7923-4580-455c-a3f1-50377a99b885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:41.646533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c7923-4580-455c-a3f1-50377a99b885 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:41.647226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c7923-4580-455c-a3f1-50377a99b885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:41.648855 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c7923-4580-455c-a3f1-50377a99b885 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:41.676337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ef9b2-ad11-46dc-8bd1-dfb6072ed27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:41.676918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ef9b2-ad11-46dc-8bd1-dfb6072ed27b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:41.677269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ef9b2-ad11-46dc-8bd1-dfb6072ed27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:41.677544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ef9b2-ad11-46dc-8bd1-dfb6072ed27b admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:41.678339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ef9b2-ad11-46dc-8bd1-dfb6072ed27b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:41.684682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ef9b2-ad11-46dc-8bd1-dfb6072ed27b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:41.694071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 887/1781] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:41.695183 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c7923-4580-455c-a3f1-50377a99b885 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:41.723440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c7923-4580-455c-a3f1-50377a99b885 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:41.754011 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b107efa-5cf2-4ef2-9726-23590c27acfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:41.755483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b107efa-5cf2-4ef2-9726-23590c27acfd admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:41.755909 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b107efa-5cf2-4ef2-9726-23590c27acfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:41.756325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b107efa-5cf2-4ef2-9726-23590c27acfd admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:41.757126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b107efa-5cf2-4ef2-9726-23590c27acfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:41.757549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b107efa-5cf2-4ef2-9726-23590c27acfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:41.759349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b107efa-5cf2-4ef2-9726-23590c27acfd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:41.766230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 895/1782] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:41 2022] POST /identity/v3/projects => generated 405 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:41.768040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ef9b2-ad11-46dc-8bd1-dfb6072ed27b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:41.797115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 888/1783] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:41.800042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b107efa-5cf2-4ef2-9726-23590c27acfd admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:41.803895 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7b107efa-5cf2-4ef2-9726-23590c27acfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:41.932851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ef9b2-ad11-46dc-8bd1-dfb6072ed27b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:41.952861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310efcb-c464-4e1e-a91d-c819f9c53f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:41.953398 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b107efa-5cf2-4ef2-9726-23590c27acfd admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:41.954627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310efcb-c464-4e1e-a91d-c819f9c53f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:41.955331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310efcb-c464-4e1e-a91d-c819f9c53f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:41.956386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310efcb-c464-4e1e-a91d-c819f9c53f81 admin admin] PATH_INFO: `/v3/projects/b1a0a97dc69a468b95302dcd036766c6/users/14c5514eed194029aebe78922ece2e63/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:41.962647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310efcb-c464-4e1e-a91d-c819f9c53f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a0a97dc69a468b95302dcd036766c6, user_id=14c5514eed194029aebe78922ece2e63, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:41.969020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310efcb-c464-4e1e-a91d-c819f9c53f81 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:42.003494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 896/1784] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:41 2022] PUT /identity/v3/projects/b1a0a97dc69a468b95302dcd036766c6/users/14c5514eed194029aebe78922ece2e63/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:42.004906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310efcb-c464-4e1e-a91d-c819f9c53f81 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:42.041096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:42.118520 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:42.147221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade5850-d67c-4722-95ac-462d87fd5c95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:42.148481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade5850-d67c-4722-95ac-462d87fd5c95 service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:42.149555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade5850-d67c-4722-95ac-462d87fd5c95 service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:42.152182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade5850-d67c-4722-95ac-462d87fd5c95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:42.268661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310efcb-c464-4e1e-a91d-c819f9c53f81 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:42.283884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b8127-8390-4636-8b91-f9ef081b7717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:42.284796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b8127-8390-4636-8b91-f9ef081b7717 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:42.285286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b8127-8390-4636-8b91-f9ef081b7717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:42.285760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b8127-8390-4636-8b91-f9ef081b7717 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:42.286748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b8127-8390-4636-8b91-f9ef081b7717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:42.288980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b8127-8390-4636-8b91-f9ef081b7717 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:42.312369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade5850-d67c-4722-95ac-462d87fd5c95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:42.314805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade5850-d67c-4722-95ac-462d87fd5c95 service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:42.369048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 889/1785] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:28:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:28:42.371454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade5850-d67c-4722-95ac-462d87fd5c95 service swift] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:42.395014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 897/1786] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:42 2022] POST /identity/v3/users => generated 303 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:42.395938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade5850-d67c-4722-95ac-462d87fd5c95 service swift] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:42.398633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138a2ff-68d8-468f-99cc-087f5dac665e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:42.399060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138a2ff-68d8-468f-99cc-087f5dac665e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:42.399346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138a2ff-68d8-468f-99cc-087f5dac665e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:42.408189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece1281-7c42-4fb2-97ca-930560437543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:42.408871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece1281-7c42-4fb2-97ca-930560437543 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:42.409273 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece1281-7c42-4fb2-97ca-930560437543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:42.409545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece1281-7c42-4fb2-97ca-930560437543 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:42.410330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece1281-7c42-4fb2-97ca-930560437543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:42.412146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece1281-7c42-4fb2-97ca-930560437543 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:42.483400 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3138a2ff-68d8-468f-99cc-087f5dac665e None None] MFA Rules not processed for user `14c5514eed194029aebe78922ece2e63`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:42.486893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 890/1787] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:42 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:42.487970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3138a2ff-68d8-468f-99cc-087f5dac665e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:42.490889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece1281-7c42-4fb2-97ca-930560437543 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:42.653457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece1281-7c42-4fb2-97ca-930560437543 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:42.679003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a288fc2-0fc0-43ea-ba4a-018032d9898e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:42.679781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a288fc2-0fc0-43ea-ba4a-018032d9898e admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:42.680057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a288fc2-0fc0-43ea-ba4a-018032d9898e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:42.680391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a288fc2-0fc0-43ea-ba4a-018032d9898e admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:42.681030 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a288fc2-0fc0-43ea-ba4a-018032d9898e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:42.684597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a288fc2-0fc0-43ea-ba4a-018032d9898e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:42.714508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 891/1788] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:42.714508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a288fc2-0fc0-43ea-ba4a-018032d9898e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:42.734811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a288fc2-0fc0-43ea-ba4a-018032d9898e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:42.745344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36168244-c67d-4fba-872a-744b312dfeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:42.745971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36168244-c67d-4fba-872a-744b312dfeaf admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:42.746326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36168244-c67d-4fba-872a-744b312dfeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:42.746507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36168244-c67d-4fba-872a-744b312dfeaf admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:42.747147 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36168244-c67d-4fba-872a-744b312dfeaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:42.749098 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36168244-c67d-4fba-872a-744b312dfeaf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:42.803090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 892/1789] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:42 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:42.803205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 898/1790] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:42.804042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36168244-c67d-4fba-872a-744b312dfeaf admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:42.804245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138a2ff-68d8-468f-99cc-087f5dac665e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:42.828094 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36168244-c67d-4fba-872a-744b312dfeaf admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:42.834665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0ca17-0ae1-4efb-928a-577329206e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:42.835303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0ca17-0ae1-4efb-928a-577329206e40 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:42.835768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0ca17-0ae1-4efb-928a-577329206e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:42.836043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0ca17-0ae1-4efb-928a-577329206e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:42.836756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0ca17-0ae1-4efb-928a-577329206e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:42.847285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0ca17-0ae1-4efb-928a-577329206e40 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:42.863113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138a2ff-68d8-468f-99cc-087f5dac665e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:42.884720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 893/1791] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:42.884720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0ca17-0ae1-4efb-928a-577329206e40 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:42.888148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be55630-c216-450a-a6f1-8660f75a75f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:42.889171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be55630-c216-450a-a6f1-8660f75a75f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:42.889454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be55630-c216-450a-a6f1-8660f75a75f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:42.889713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be55630-c216-450a-a6f1-8660f75a75f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:42.890374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be55630-c216-450a-a6f1-8660f75a75f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:42.892116 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be55630-c216-450a-a6f1-8660f75a75f7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:42.894475 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3ba0ca17-0ae1-4efb-928a-577329206e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:42.921266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 899/1792] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:42.923243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be55630-c216-450a-a6f1-8660f75a75f7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:42.956102 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be55630-c216-450a-a6f1-8660f75a75f7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:42.969432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa984d6-067c-4a45-bb42-7790e7d440df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:42.970136 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa984d6-067c-4a45-bb42-7790e7d440df admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:42.970413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa984d6-067c-4a45-bb42-7790e7d440df admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:42.970688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa984d6-067c-4a45-bb42-7790e7d440df admin admin] PATH_INFO: `/v3/projects/2a1c1161a8874b5886c985009d1d4ff4/users/4397170e783047629e56e48278019df1/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:43.003131 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:43.018145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa984d6-067c-4a45-bb42-7790e7d440df admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a1c1161a8874b5886c985009d1d4ff4, user_id=4397170e783047629e56e48278019df1, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:43.022958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa984d6-067c-4a45-bb42-7790e7d440df admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:43.062449 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0ca17-0ae1-4efb-928a-577329206e40 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:43.063990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 900/1793] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:42 2022] PUT /identity/v3/projects/2a1c1161a8874b5886c985009d1d4ff4/users/4397170e783047629e56e48278019df1/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:43.064929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa984d6-067c-4a45-bb42-7790e7d440df admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:43.129531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:43.163419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa984d6-067c-4a45-bb42-7790e7d440df admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:43.174331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da1f9a-0ad2-475f-a68f-21fe2a9ab0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:43.175205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da1f9a-0ad2-475f-a68f-21fe2a9ab0bd admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:43.175697 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da1f9a-0ad2-475f-a68f-21fe2a9ab0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:43.175977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da1f9a-0ad2-475f-a68f-21fe2a9ab0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:43.176805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da1f9a-0ad2-475f-a68f-21fe2a9ab0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:43.178855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da1f9a-0ad2-475f-a68f-21fe2a9ab0bd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:43.205607 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 901/1794] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:43.205607 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14077b-d43b-433d-84c7-63e6062d7e57 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:43.205607 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14077b-d43b-433d-84c7-63e6062d7e57 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:43.206249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14077b-d43b-433d-84c7-63e6062d7e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:43.210857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a61b9-404b-4930-962e-c9e51758d3c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:43.213663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a61b9-404b-4930-962e-c9e51758d3c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:43.214176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a61b9-404b-4930-962e-c9e51758d3c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:43.214641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a61b9-404b-4930-962e-c9e51758d3c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:43.220431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-981a61b9-404b-4930-962e-c9e51758d3c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:43.237030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0a14077b-d43b-433d-84c7-63e6062d7e57 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:43.238659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0a14077b-d43b-433d-84c7-63e6062d7e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:43.253620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 902/1795] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:43.255006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14077b-d43b-433d-84c7-63e6062d7e57 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:43.269200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14077b-d43b-433d-84c7-63e6062d7e57 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:43.276045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc13cba-2528-43c4-b288-e7d214dcc404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:43.276968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc13cba-2528-43c4-b288-e7d214dcc404 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:43.277338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc13cba-2528-43c4-b288-e7d214dcc404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:43.277675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc13cba-2528-43c4-b288-e7d214dcc404 admin admin] PATH_INFO: `/v3/projects/6f8e5da07136490988f2be09153304dc/users/f2fce365cc8b42a4877957d48771cd1f/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:43.303405 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc13cba-2528-43c4-b288-e7d214dcc404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f8e5da07136490988f2be09153304dc, user_id=f2fce365cc8b42a4877957d48771cd1f, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:43.307179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc13cba-2528-43c4-b288-e7d214dcc404 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:43.329000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 903/1796] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:43 2022] PUT /identity/v3/projects/6f8e5da07136490988f2be09153304dc/users/f2fce365cc8b42a4877957d48771cd1f/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:43.339118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b3fa7-ad46-4127-b892-4b4c829b1011 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:43.339643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b3fa7-ad46-4127-b892-4b4c829b1011 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:43.340042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b3fa7-ad46-4127-b892-4b4c829b1011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:43.403056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a61b9-404b-4930-962e-c9e51758d3c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:43.405312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a61b9-404b-4930-962e-c9e51758d3c1 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:43.406510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f99b3fa7-ad46-4127-b892-4b4c829b1011 None None] MFA Rules not processed for user `f2fce365cc8b42a4877957d48771cd1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:43.407834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f99b3fa7-ad46-4127-b892-4b4c829b1011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:43.439526 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 894/1797] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:43.488037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a61b9-404b-4930-962e-c9e51758d3c1 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:43.489589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-981a61b9-404b-4930-962e-c9e51758d3c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:43.548594 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 904/1798] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:43.553125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b3fa7-ad46-4127-b892-4b4c829b1011 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:43.673794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a61b9-404b-4930-962e-c9e51758d3c1 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:43.715504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b3fa7-ad46-4127-b892-4b4c829b1011 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:43.746017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558dfaa-b349-4837-99da-1d650b3e935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:43.746558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558dfaa-b349-4837-99da-1d650b3e935b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:43.746834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558dfaa-b349-4837-99da-1d650b3e935b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:43.747105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558dfaa-b349-4837-99da-1d650b3e935b admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:43.757306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558dfaa-b349-4837-99da-1d650b3e935b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:43.761097 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558dfaa-b349-4837-99da-1d650b3e935b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:43.855534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 905/1799] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:43 2022] POST /identity/v3/projects => generated 365 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:28:43.860404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558dfaa-b349-4837-99da-1d650b3e935b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:43.923093 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0fd62-9d80-4cda-ade2-2e4df2d9d957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:43.925134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0fd62-9d80-4cda-ade2-2e4df2d9d957 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:43.927448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0fd62-9d80-4cda-ade2-2e4df2d9d957 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:43.927790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0fd62-9d80-4cda-ade2-2e4df2d9d957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:43.939217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:43.993829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0fd62-9d80-4cda-ade2-2e4df2d9d957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:44.005490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0fd62-9d80-4cda-ade2-2e4df2d9d957 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:44.052487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 895/1800] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:44.057204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8968a-4962-4157-a3d4-bbc42f927f63 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:44.057467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8968a-4962-4157-a3d4-bbc42f927f63 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:44.057765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8968a-4962-4157-a3d4-bbc42f927f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:44.072995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558dfaa-b349-4837-99da-1d650b3e935b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:44.082235 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:44.083434 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a8142-c95d-4a8e-bab8-8e01715f745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:44.084075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a8142-c95d-4a8e-bab8-8e01715f745b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:44.084353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a8142-c95d-4a8e-bab8-8e01715f745b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:44.084618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a8142-c95d-4a8e-bab8-8e01715f745b admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:44.086459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a8142-c95d-4a8e-bab8-8e01715f745b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:44.092368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a8142-c95d-4a8e-bab8-8e01715f745b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:44.108413 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-bcf8968a-4962-4157-a3d4-bbc42f927f63 None None] MFA Rules not processed for user `c9360be7ee9e4739b20c59955d28da47`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:44.112488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bcf8968a-4962-4157-a3d4-bbc42f927f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:44.147828 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 906/1801] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:43 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:28:44.149671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a8142-c95d-4a8e-bab8-8e01715f745b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:44.196161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a8142-c95d-4a8e-bab8-8e01715f745b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:44.219602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab462abc-4f93-44d7-af20-99f1f4c29f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:44.220145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab462abc-4f93-44d7-af20-99f1f4c29f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:44.220415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab462abc-4f93-44d7-af20-99f1f4c29f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:44.220681 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab462abc-4f93-44d7-af20-99f1f4c29f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:44.221423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab462abc-4f93-44d7-af20-99f1f4c29f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:44.223423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab462abc-4f93-44d7-af20-99f1f4c29f0e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:44.264480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 907/1802] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:44.268490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab462abc-4f93-44d7-af20-99f1f4c29f0e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:44.287370 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab462abc-4f93-44d7-af20-99f1f4c29f0e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:44.297740 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 896/1803] 10.222.0.66 () {58 vars in 1015 bytes} [Sat Sep 17 04:28:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:44.301185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97492923-fe97-4fe0-9e8e-279627f85d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:44.301889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97492923-fe97-4fe0-9e8e-279627f85d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:44.302169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97492923-fe97-4fe0-9e8e-279627f85d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:44.302440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97492923-fe97-4fe0-9e8e-279627f85d36 admin admin] PATH_INFO: `/v3/projects/680a6904139b4a68967d055bba9f9447/users/c05aa65bf77e48818a0340232d4e72f7/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:44.326758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8968a-4962-4157-a3d4-bbc42f927f63 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:44.329310 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bcf8968a-4962-4157-a3d4-bbc42f927f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:44.371385 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8968a-4962-4157-a3d4-bbc42f927f63 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:44.382690 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97492923-fe97-4fe0-9e8e-279627f85d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=680a6904139b4a68967d055bba9f9447, user_id=c05aa65bf77e48818a0340232d4e72f7, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:44.387250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97492923-fe97-4fe0-9e8e-279627f85d36 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:44.392012 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cc09b-7f1c-40d5-8476-13956b84568f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:44.392680 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cc09b-7f1c-40d5-8476-13956b84568f service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:44.392900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cc09b-7f1c-40d5-8476-13956b84568f service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:44.393249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cc09b-7f1c-40d5-8476-13956b84568f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:44.405905 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cc09b-7f1c-40d5-8476-13956b84568f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:44.408582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cc09b-7f1c-40d5-8476-13956b84568f service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:44.433827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 908/1804] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:44 2022] PUT /identity/v3/projects/680a6904139b4a68967d055bba9f9447/users/c05aa65bf77e48818a0340232d4e72f7/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:44.434813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97492923-fe97-4fe0-9e8e-279627f85d36 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:44.439698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-97492923-fe97-4fe0-9e8e-279627f85d36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:44.475142 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 897/1805] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:44.477587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cc09b-7f1c-40d5-8476-13956b84568f service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:44.532676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cc09b-7f1c-40d5-8476-13956b84568f service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:44.704273 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85600807-c6c4-4834-9dad-8f1ed07be455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:44.704798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85600807-c6c4-4834-9dad-8f1ed07be455 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:44.705208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85600807-c6c4-4834-9dad-8f1ed07be455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:44.706953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85600807-c6c4-4834-9dad-8f1ed07be455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:44.707732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85600807-c6c4-4834-9dad-8f1ed07be455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:44.709515 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85600807-c6c4-4834-9dad-8f1ed07be455 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:44.734955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 898/1806] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:44.739750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85600807-c6c4-4834-9dad-8f1ed07be455 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:44.753812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97492923-fe97-4fe0-9e8e-279627f85d36 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:44.757760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85600807-c6c4-4834-9dad-8f1ed07be455 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:44.763473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddbfa4-1f1a-45d3-947b-a88b951aca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:44.764239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddbfa4-1f1a-45d3-947b-a88b951aca87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:44.764519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddbfa4-1f1a-45d3-947b-a88b951aca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:44.765865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddbfa4-1f1a-45d3-947b-a88b951aca87 admin admin] PATH_INFO: `/v3/projects/680a6904139b4a68967d055bba9f9447/users/c05aa65bf77e48818a0340232d4e72f7/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:44.770441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:44.773195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:44.773722 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:44.774427 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] PATH_INFO: `/v3/projects/5af63acf1c584cb4a4a55e9e53f05068` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:44.777860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5af63acf1c584cb4a4a55e9e53f05068)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:44.778197 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddbfa4-1f1a-45d3-947b-a88b951aca87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=680a6904139b4a68967d055bba9f9447, user_id=c05aa65bf77e48818a0340232d4e72f7, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:44.782618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:44.783811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddbfa4-1f1a-45d3-947b-a88b951aca87 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:44.859394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 899/1807] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:44 2022] PUT /identity/v3/projects/680a6904139b4a68967d055bba9f9447/users/c05aa65bf77e48818a0340232d4e72f7/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:44.874124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7218c-d9b6-4e0c-9e60-9ae8937fa93a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:44.874953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7218c-d9b6-4e0c-9e60-9ae8937fa93a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:44.875232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7218c-d9b6-4e0c-9e60-9ae8937fa93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:44.945466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5af63acf1c584cb4a4a55e9e53f05068', 'request_id': 'req-d3198401-2555-4043-8b06-2146ba826610'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:44.953474 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-8bf7218c-d9b6-4e0c-9e60-9ae8937fa93a None None] MFA Rules not processed for user `c05aa65bf77e48818a0340232d4e72f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:44.956396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8bf7218c-d9b6-4e0c-9e60-9ae8937fa93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:44.961562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5af63acf1c584cb4a4a55e9e53f05068', 'request_id': 'req-d3198401-2555-4043-8b06-2146ba826610'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:44.986100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5af63acf1c584cb4a4a55e9e53f05068', 'request_id': 'req-d3198401-2555-4043-8b06-2146ba826610'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.024688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] The token cache is being invalidate because project 5af63acf1c584cb4a4a55e9e53f05068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:45.024992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.029236 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.035699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 909/1808] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:44 2022] DELETE /identity/v3/projects/5af63acf1c584cb4a4a55e9e53f05068 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:45.041075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:45.044024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:45.075378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:45.187733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 900/1809] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:45.202376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7218c-d9b6-4e0c-9e60-9ae8937fa93a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:45.218952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8bf7218c-d9b6-4e0c-9e60-9ae8937fa93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:45.258407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:45.332833 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3198401-2555-4043-8b06-2146ba826610 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:45.366747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:45.368355 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:45.369265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:45.370006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] PATH_INFO: `/v3/users/a37f484345f34966b28d5501cb9b86a6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:45.372487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a37f484345f34966b28d5501cb9b86a6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:45.376191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:45.430277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7218c-d9b6-4e0c-9e60-9ae8937fa93a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:45.453716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae19615-64ad-4bf6-876b-fe78aaabb6f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:45.454810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae19615-64ad-4bf6-876b-fe78aaabb6f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:45.455348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae19615-64ad-4bf6-876b-fe78aaabb6f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:45.455852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae19615-64ad-4bf6-876b-fe78aaabb6f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:45.472269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a37f484345f34966b28d5501cb9b86a6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.482238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae19615-64ad-4bf6-876b-fe78aaabb6f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:45.484013 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae19615-64ad-4bf6-876b-fe78aaabb6f7 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:45.494837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a37f484345f34966b28d5501cb9b86a6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.496806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a37f484345f34966b28d5501cb9b86a6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.504402 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 901/1810] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:45.505481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae19615-64ad-4bf6-876b-fe78aaabb6f7 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:45.507421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7ae19615-64ad-4bf6-876b-fe78aaabb6f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:45.539571 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a37f484345f34966b28d5501cb9b86a6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.541801 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae19615-64ad-4bf6-876b-fe78aaabb6f7 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:45.554484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2d52f-d9ab-472d-bf88-6aa466635465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:45.555524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2d52f-d9ab-472d-bf88-6aa466635465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:45.556014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2d52f-d9ab-472d-bf88-6aa466635465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:45.556495 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2d52f-d9ab-472d-bf88-6aa466635465 admin admin] PATH_INFO: `/v3/users/31092239f73d4bc69bb2d286af03811a` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:45.557911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2d52f-d9ab-472d-bf88-6aa466635465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31092239f73d4bc69bb2d286af03811a)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:45.560130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2d52f-d9ab-472d-bf88-6aa466635465 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:45.600226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a37f484345f34966b28d5501cb9b86a6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.621675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 910/1811] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:45 2022] DELETE /identity/v3/users/a37f484345f34966b28d5501cb9b86a6 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:45.715010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a9e2d52f-d9ab-472d-bf88-6aa466635465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31092239f73d4bc69bb2d286af03811a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.715980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:45.718677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:45.738611 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a9e2d52f-d9ab-472d-bf88-6aa466635465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31092239f73d4bc69bb2d286af03811a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.739608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a9e2d52f-d9ab-472d-bf88-6aa466635465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31092239f73d4bc69bb2d286af03811a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.760263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a9e2d52f-d9ab-472d-bf88-6aa466635465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31092239f73d4bc69bb2d286af03811a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.788964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a9e2d52f-d9ab-472d-bf88-6aa466635465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31092239f73d4bc69bb2d286af03811a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:45.871592 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 902/1812] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:45 2022] DELETE /identity/v3/users/31092239f73d4bc69bb2d286af03811a => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:45.874837 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63710d-11d2-4d34-91e8-2aab7bc9689a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:45.900476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63710d-11d2-4d34-91e8-2aab7bc9689a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:45.900476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63710d-11d2-4d34-91e8-2aab7bc9689a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:45.938783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd3ddb-98fa-4a97-866d-3c31449bb60f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:45.962343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ef63710d-11d2-4d34-91e8-2aab7bc9689a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:45.973767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ef63710d-11d2-4d34-91e8-2aab7bc9689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:46.035118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:46.059887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988ecdb-3040-4f8b-a162-6783d5902c92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:46.062145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988ecdb-3040-4f8b-a162-6783d5902c92 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:46.062375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988ecdb-3040-4f8b-a162-6783d5902c92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:46.062774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988ecdb-3040-4f8b-a162-6783d5902c92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:46.071461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8988ecdb-3040-4f8b-a162-6783d5902c92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:46.176609 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 903/1813] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:46.199798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988ecdb-3040-4f8b-a162-6783d5902c92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:46.201414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988ecdb-3040-4f8b-a162-6783d5902c92 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:46.218109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63710d-11d2-4d34-91e8-2aab7bc9689a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:46.220473 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 911/1814] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:46.230062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ef63710d-11d2-4d34-91e8-2aab7bc9689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:46.303077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:46.385930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63710d-11d2-4d34-91e8-2aab7bc9689a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:46.390810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96f890-865e-4f05-81b1-c47749b9d339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:46.391783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96f890-865e-4f05-81b1-c47749b9d339 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:46.391937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96f890-865e-4f05-81b1-c47749b9d339 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:46.392377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96f890-865e-4f05-81b1-c47749b9d339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:46.404741 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96f890-865e-4f05-81b1-c47749b9d339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:46.411897 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96f890-865e-4f05-81b1-c47749b9d339 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:46.432612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 904/1815] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:28:47.840229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424fda1-8049-4b56-9829-7cfea38a841b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:47.840785 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424fda1-8049-4b56-9829-7cfea38a841b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:47.840785 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424fda1-8049-4b56-9829-7cfea38a841b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:47.842147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 912/1816] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:28:47.883353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed482523-807f-4551-a04c-446358a43707 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:47.884021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed482523-807f-4551-a04c-446358a43707 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:47.884021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed482523-807f-4551-a04c-446358a43707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:47.905293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ed482523-807f-4551-a04c-446358a43707 None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:47.907292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ed482523-807f-4551-a04c-446358a43707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:48.060824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 905/1817] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:28:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:28:48.093500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424fda1-8049-4b56-9829-7cfea38a841b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:48.093500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3424fda1-8049-4b56-9829-7cfea38a841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:48.110184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424fda1-8049-4b56-9829-7cfea38a841b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:48.115276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:48.115920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:48.116183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:48.116445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] PATH_INFO: `/v3/projects/ef1382625553428eb1f818c507cafa7e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:48.117605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef1382625553428eb1f818c507cafa7e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:48.119530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:48.183831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef1382625553428eb1f818c507cafa7e', 'request_id': 'req-215ab405-ed61-4453-9a61-b72329877d0c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.193688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef1382625553428eb1f818c507cafa7e', 'request_id': 'req-215ab405-ed61-4453-9a61-b72329877d0c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.201458 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef1382625553428eb1f818c507cafa7e', 'request_id': 'req-215ab405-ed61-4453-9a61-b72329877d0c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.215743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] The token cache is being invalidate because project ef1382625553428eb1f818c507cafa7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:48.215743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.216470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.218328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 913/1818] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:48 2022] DELETE /identity/v3/projects/ef1382625553428eb1f818c507cafa7e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:48.223802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef12e-54a2-45e9-a034-13af3efefa47 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:48.224439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef12e-54a2-45e9-a034-13af3efefa47 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:48.224988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef12e-54a2-45e9-a034-13af3efefa47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:48.263004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b0cef12e-54a2-45e9-a034-13af3efefa47 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:48.266365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b0cef12e-54a2-45e9-a034-13af3efefa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:48.294648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:48.300103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:48.403264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 906/1819] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:48.404668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cef12e-54a2-45e9-a034-13af3efefa47 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:48.406980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b0cef12e-54a2-45e9-a034-13af3efefa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:48.436546 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ab405-ed61-4453-9a61-b72329877d0c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:48.442349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:48.443002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:48.443273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:48.443639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848 admin admin] PATH_INFO: `/v3/projects/b29d0d22659b47b1a14d249a49e94c9b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:48.445502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b29d0d22659b47b1a14d249a49e94c9b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:48.447685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:48.475191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:48.486780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:48.511569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cef12e-54a2-45e9-a034-13af3efefa47 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:48.513276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b29d0d22659b47b1a14d249a49e94c9b', 'request_id': 'req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.525610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b29d0d22659b47b1a14d249a49e94c9b', 'request_id': 'req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.534135 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b29d0d22659b47b1a14d249a49e94c9b', 'request_id': 'req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.562127 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848 admin admin] The token cache is being invalidate because project b29d0d22659b47b1a14d249a49e94c9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:48.562423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.563502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bcebc3a1-7fbd-4316-a559-a9e8a5f5a848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.568472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 914/1820] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:48 2022] DELETE /identity/v3/projects/b29d0d22659b47b1a14d249a49e94c9b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:48.569962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d9940-c0ec-4e52-a0a7-3cf798ea3c55 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:48.570245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d9940-c0ec-4e52-a0a7-3cf798ea3c55 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:48.570653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d9940-c0ec-4e52-a0a7-3cf798ea3c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:48.601747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-361d9940-c0ec-4e52-a0a7-3cf798ea3c55 None None] MFA Rules not processed for user `4397170e783047629e56e48278019df1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:48.603452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-361d9940-c0ec-4e52-a0a7-3cf798ea3c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:48.622754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4f484-e4d4-42b9-94d9-0995e4588b27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:48.624519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4f484-e4d4-42b9-94d9-0995e4588b27 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:48.625015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4f484-e4d4-42b9-94d9-0995e4588b27 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:48.625704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4f484-e4d4-42b9-94d9-0995e4588b27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:48.630475 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-34e4f484-e4d4-42b9-94d9-0995e4588b27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:48.644862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4f484-e4d4-42b9-94d9-0995e4588b27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:48.646760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4f484-e4d4-42b9-94d9-0995e4588b27 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:48.692266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 907/1821] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:48.695688 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc35b8-069d-48ad-8b95-e0a78e8a77b0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:48.695688 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc35b8-069d-48ad-8b95-e0a78e8a77b0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:48.696196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc35b8-069d-48ad-8b95-e0a78e8a77b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:48.719233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 915/1822] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:48 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:48.719233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d9940-c0ec-4e52-a0a7-3cf798ea3c55 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:48.720567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-361d9940-c0ec-4e52-a0a7-3cf798ea3c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:48.725752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c8dc35b8-069d-48ad-8b95-e0a78e8a77b0 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:48.727551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c8dc35b8-069d-48ad-8b95-e0a78e8a77b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:48.846421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 908/1823] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:48.847762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb628d-7af2-470e-b598-dc0a57ef73a5 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:48.848056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb628d-7af2-470e-b598-dc0a57ef73a5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:48.848329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb628d-7af2-470e-b598-dc0a57ef73a5 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:48.849875 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 909/1824] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:28:48.850574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb628d-7af2-470e-b598-dc0a57ef73a5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:48.852608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6fcb628d-7af2-470e-b598-dc0a57ef73a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:48.858937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d9940-c0ec-4e52-a0a7-3cf798ea3c55 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:48.866254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89505fdd-9cee-4641-a3df-639d36734d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:48.866910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89505fdd-9cee-4641-a3df-639d36734d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:48.867212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89505fdd-9cee-4641-a3df-639d36734d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:48.867450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89505fdd-9cee-4641-a3df-639d36734d37 admin admin] PATH_INFO: `/v3/users/c6b814ffce8c41268e260ec60e81f81d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:48.868572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89505fdd-9cee-4641-a3df-639d36734d37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6b814ffce8c41268e260ec60e81f81d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:48.870250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb628d-7af2-470e-b598-dc0a57ef73a5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:48.870552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89505fdd-9cee-4641-a3df-639d36734d37 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:48.875337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736eac92-2044-4e18-b9a4-2eb38e8e000a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:48.876125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736eac92-2044-4e18-b9a4-2eb38e8e000a admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:48.877122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736eac92-2044-4e18-b9a4-2eb38e8e000a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:48.877122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736eac92-2044-4e18-b9a4-2eb38e8e000a admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:48.877489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736eac92-2044-4e18-b9a4-2eb38e8e000a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:48.879398 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736eac92-2044-4e18-b9a4-2eb38e8e000a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:48.915374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 910/1825] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:48 2022] POST /identity/v3/projects => generated 401 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:48.915374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736eac92-2044-4e18-b9a4-2eb38e8e000a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:48.919152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:48.928573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89505fdd-9cee-4641-a3df-639d36734d37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6b814ffce8c41268e260ec60e81f81d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.943008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89505fdd-9cee-4641-a3df-639d36734d37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6b814ffce8c41268e260ec60e81f81d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.949606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89505fdd-9cee-4641-a3df-639d36734d37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6b814ffce8c41268e260ec60e81f81d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.962779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89505fdd-9cee-4641-a3df-639d36734d37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6b814ffce8c41268e260ec60e81f81d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.988240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89505fdd-9cee-4641-a3df-639d36734d37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6b814ffce8c41268e260ec60e81f81d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:48.993223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736eac92-2044-4e18-b9a4-2eb38e8e000a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:48.994284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 916/1826] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:48 2022] DELETE /identity/v3/users/c6b814ffce8c41268e260ec60e81f81d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:48.995583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7457a7d-92e4-4600-b82b-5c737db1f2ee None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:48.995873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7457a7d-92e4-4600-b82b-5c737db1f2ee None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:48.996148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7457a7d-92e4-4600-b82b-5c737db1f2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:48.998961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:49.018240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e7457a7d-92e4-4600-b82b-5c737db1f2ee None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:49.019470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e7457a7d-92e4-4600-b82b-5c737db1f2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:49.061134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a637103-e980-451b-8065-1c0d97b14cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:49.061134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a637103-e980-451b-8065-1c0d97b14cce admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.061134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a637103-e980-451b-8065-1c0d97b14cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.061134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a637103-e980-451b-8065-1c0d97b14cce admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.061757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a637103-e980-451b-8065-1c0d97b14cce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:49.061757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a637103-e980-451b-8065-1c0d97b14cce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:49.063435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a637103-e980-451b-8065-1c0d97b14cce admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:49.078102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 911/1827] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:49.079514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a637103-e980-451b-8065-1c0d97b14cce admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:49.082145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0a637103-e980-451b-8065-1c0d97b14cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:49.096613 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a637103-e980-451b-8065-1c0d97b14cce admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:49.109927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 917/1828] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:28:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:49.111055 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7457a7d-92e4-4600-b82b-5c737db1f2ee None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:49.112565 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e7457a7d-92e4-4600-b82b-5c737db1f2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:49.128687 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffa323-9b5f-4bf1-8f1e-629c2cb500e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:49.129441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7457a7d-92e4-4600-b82b-5c737db1f2ee None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:49.130308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffa323-9b5f-4bf1-8f1e-629c2cb500e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.131916 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffa323-9b5f-4bf1-8f1e-629c2cb500e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.132259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffa323-9b5f-4bf1-8f1e-629c2cb500e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.133000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ffa323-9b5f-4bf1-8f1e-629c2cb500e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:49.135319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a276cd-be5f-4d8b-a58d-6cb8ec14e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:49.136094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a276cd-be5f-4d8b-a58d-6cb8ec14e3af admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.136504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a276cd-be5f-4d8b-a58d-6cb8ec14e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.136914 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a276cd-be5f-4d8b-a58d-6cb8ec14e3af admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.137715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a276cd-be5f-4d8b-a58d-6cb8ec14e3af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:49.139829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ffa323-9b5f-4bf1-8f1e-629c2cb500e2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:49.140470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a276cd-be5f-4d8b-a58d-6cb8ec14e3af admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:49.182118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 912/1829] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:49 2022] POST /identity/v3/users => generated 307 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:49.186258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffa323-9b5f-4bf1-8f1e-629c2cb500e2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:49.189032 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 918/1830] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:49 2022] POST /identity/v3/projects => generated 371 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:49.190256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a276cd-be5f-4d8b-a58d-6cb8ec14e3af admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:49.190981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a1ffa323-9b5f-4bf1-8f1e-629c2cb500e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:49.244480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffa323-9b5f-4bf1-8f1e-629c2cb500e2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:49.327395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a276cd-be5f-4d8b-a58d-6cb8ec14e3af admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:49.340768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922a97c-c651-4518-b2bd-32af6af377f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:49.341525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922a97c-c651-4518-b2bd-32af6af377f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.341972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922a97c-c651-4518-b2bd-32af6af377f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.342255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922a97c-c651-4518-b2bd-32af6af377f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.343197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922a97c-c651-4518-b2bd-32af6af377f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:49.345592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922a97c-c651-4518-b2bd-32af6af377f1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:49.362011 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 919/1831] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:49.362865 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922a97c-c651-4518-b2bd-32af6af377f1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:49.377303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6973942-2e8b-48e0-95b3-006535ee2b08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:49.378243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6973942-2e8b-48e0-95b3-006535ee2b08 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.378704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6973942-2e8b-48e0-95b3-006535ee2b08 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.379106 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6973942-2e8b-48e0-95b3-006535ee2b08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.387323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922a97c-c651-4518-b2bd-32af6af377f1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:49.391551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6973942-2e8b-48e0-95b3-006535ee2b08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:49.394609 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6973942-2e8b-48e0-95b3-006535ee2b08 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:49.404195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc57bf2-4700-490a-9637-be97cb1e907c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:49.405003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc57bf2-4700-490a-9637-be97cb1e907c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.405769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc57bf2-4700-490a-9637-be97cb1e907c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.417443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc57bf2-4700-490a-9637-be97cb1e907c admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.418610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc57bf2-4700-490a-9637-be97cb1e907c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:49.421983 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc57bf2-4700-490a-9637-be97cb1e907c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:49.432833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 913/1832] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:49.435292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6973942-2e8b-48e0-95b3-006535ee2b08 service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:49.463759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6973942-2e8b-48e0-95b3-006535ee2b08 service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:49.478597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92cc56-6e8d-4ed8-b006-ede960ff967b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:49.479454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92cc56-6e8d-4ed8-b006-ede960ff967b admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.479889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92cc56-6e8d-4ed8-b006-ede960ff967b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.480311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92cc56-6e8d-4ed8-b006-ede960ff967b admin admin] PATH_INFO: `/v3/projects/5d757ddd957d44008050826243085995/users/5aac7f666e9345ca8f51fe7c7f9f74a8/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.486332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 920/1833] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:49 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:49.498096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc57bf2-4700-490a-9637-be97cb1e907c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:49.524504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92cc56-6e8d-4ed8-b006-ede960ff967b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d757ddd957d44008050826243085995, user_id=5aac7f666e9345ca8f51fe7c7f9f74a8, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:49.531573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92cc56-6e8d-4ed8-b006-ede960ff967b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:49.544148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc57bf2-4700-490a-9637-be97cb1e907c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:49.552076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa55bd-bb49-41f4-a849-f9f05e2a59eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:49.554056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa55bd-bb49-41f4-a849-f9f05e2a59eb admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.554813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa55bd-bb49-41f4-a849-f9f05e2a59eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.555234 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa55bd-bb49-41f4-a849-f9f05e2a59eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.556510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa55bd-bb49-41f4-a849-f9f05e2a59eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:49.560660 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa55bd-bb49-41f4-a849-f9f05e2a59eb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:49.562838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:49.564037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 914/1834] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:49 2022] PUT /identity/v3/projects/5d757ddd957d44008050826243085995/users/5aac7f666e9345ca8f51fe7c7f9f74a8/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:49.567914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92cc56-6e8d-4ed8-b006-ede960ff967b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:49.587041 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 921/1835] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:49.594536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa55bd-bb49-41f4-a849-f9f05e2a59eb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:49.614171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa55bd-bb49-41f4-a849-f9f05e2a59eb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:49.702455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:49.715594 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f3315-04d9-4b6c-8a39-35970fbdb956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:49.716324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f3315-04d9-4b6c-8a39-35970fbdb956 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.716808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f3315-04d9-4b6c-8a39-35970fbdb956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.717549 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f3315-04d9-4b6c-8a39-35970fbdb956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.718214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f3315-04d9-4b6c-8a39-35970fbdb956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:49.720573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f3315-04d9-4b6c-8a39-35970fbdb956 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:49.732474 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92cc56-6e8d-4ed8-b006-ede960ff967b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:49.738585 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0ab90-afad-4687-a767-e0ce1c305fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:49.739610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0ab90-afad-4687-a767-e0ce1c305fca admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.740245 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0ab90-afad-4687-a767-e0ce1c305fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.740668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0ab90-afad-4687-a767-e0ce1c305fca admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.740877 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 922/1836] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:49.741664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0ab90-afad-4687-a767-e0ce1c305fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:49.743459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0ab90-afad-4687-a767-e0ce1c305fca admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:49.744869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f3315-04d9-4b6c-8a39-35970fbdb956 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:49.757346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f3315-04d9-4b6c-8a39-35970fbdb956 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:49.764681 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 915/1837] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:49.766944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c3081-a088-4375-9f90-4bc47b6af1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:49.767813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c3081-a088-4375-9f90-4bc47b6af1ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.768253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c3081-a088-4375-9f90-4bc47b6af1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.768669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c3081-a088-4375-9f90-4bc47b6af1ff admin admin] PATH_INFO: `/v3/projects/0fa33c59d868470aa3976580c77bfe40/users/ad7c3b2e18fd4edab09dc40efb02a377/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.775918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0ab90-afad-4687-a767-e0ce1c305fca admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:49.797805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c3081-a088-4375-9f90-4bc47b6af1ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fa33c59d868470aa3976580c77bfe40, user_id=ad7c3b2e18fd4edab09dc40efb02a377, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:49.807052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c3081-a088-4375-9f90-4bc47b6af1ff admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:49.810781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0ab90-afad-4687-a767-e0ce1c305fca admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:49.826249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9246442-033e-4b47-89ce-94a63917e5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:49.827237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9246442-033e-4b47-89ce-94a63917e5df admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.827518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9246442-033e-4b47-89ce-94a63917e5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.827874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9246442-033e-4b47-89ce-94a63917e5df admin admin] PATH_INFO: `/v3/projects/5d757ddd957d44008050826243085995/users/5aac7f666e9345ca8f51fe7c7f9f74a8/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.845187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9246442-033e-4b47-89ce-94a63917e5df admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d757ddd957d44008050826243085995, user_id=5aac7f666e9345ca8f51fe7c7f9f74a8, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:49.849188 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9246442-033e-4b47-89ce-94a63917e5df admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:49.856482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 923/1838] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:49 2022] PUT /identity/v3/projects/0fa33c59d868470aa3976580c77bfe40/users/ad7c3b2e18fd4edab09dc40efb02a377/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:49.872590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274182b6-865c-4dd5-ae1a-fde113c8438e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:49.873343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274182b6-865c-4dd5-ae1a-fde113c8438e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:49.873343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274182b6-865c-4dd5-ae1a-fde113c8438e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:49.877925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 916/1839] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:49 2022] PUT /identity/v3/projects/5d757ddd957d44008050826243085995/users/5aac7f666e9345ca8f51fe7c7f9f74a8/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:49.878952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9246442-033e-4b47-89ce-94a63917e5df admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:49.880697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c9246442-033e-4b47-89ce-94a63917e5df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:49.940497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-274182b6-865c-4dd5-ae1a-fde113c8438e None None] MFA Rules not processed for user `ad7c3b2e18fd4edab09dc40efb02a377`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:49.941747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-274182b6-865c-4dd5-ae1a-fde113c8438e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:49.990989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9246442-033e-4b47-89ce-94a63917e5df admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:50.014830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091326e0-78df-49c5-ba96-6756d360ec84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:50.015597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091326e0-78df-49c5-ba96-6756d360ec84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:50.015984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091326e0-78df-49c5-ba96-6756d360ec84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:50.016348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091326e0-78df-49c5-ba96-6756d360ec84 admin admin] PATH_INFO: `/v3/projects/3d8d7d6a67644cb19dd5318b39ebc85e` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:50.017604 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091326e0-78df-49c5-ba96-6756d360ec84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d8d7d6a67644cb19dd5318b39ebc85e)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:50.019852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091326e0-78df-49c5-ba96-6756d360ec84 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:50.072651 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 924/1840] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:50.079204 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274182b6-865c-4dd5-ae1a-fde113c8438e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:50.087353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-091326e0-78df-49c5-ba96-6756d360ec84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d8d7d6a67644cb19dd5318b39ebc85e', 'request_id': 'req-091326e0-78df-49c5-ba96-6756d360ec84'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:50.098239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274182b6-865c-4dd5-ae1a-fde113c8438e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:50.105201 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-091326e0-78df-49c5-ba96-6756d360ec84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d8d7d6a67644cb19dd5318b39ebc85e', 'request_id': 'req-091326e0-78df-49c5-ba96-6756d360ec84'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:50.116182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-091326e0-78df-49c5-ba96-6756d360ec84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d8d7d6a67644cb19dd5318b39ebc85e', 'request_id': 'req-091326e0-78df-49c5-ba96-6756d360ec84'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:50.133308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-091326e0-78df-49c5-ba96-6756d360ec84 admin admin] The token cache is being invalidate because project 3d8d7d6a67644cb19dd5318b39ebc85e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:50.133694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-091326e0-78df-49c5-ba96-6756d360ec84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:50.134455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-091326e0-78df-49c5-ba96-6756d360ec84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:50.137131 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 917/1841] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:49 2022] DELETE /identity/v3/projects/3d8d7d6a67644cb19dd5318b39ebc85e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:50.143886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062da08d-eb06-470e-8eff-cb43ab4dc063 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:50.144322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062da08d-eb06-470e-8eff-cb43ab4dc063 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:50.144739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062da08d-eb06-470e-8eff-cb43ab4dc063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:50.176374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:50.186822 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d567b4-e5ff-4e91-aef7-37d30fe8e3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:50.188218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d567b4-e5ff-4e91-aef7-37d30fe8e3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:50.188492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d567b4-e5ff-4e91-aef7-37d30fe8e3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:50.190595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d567b4-e5ff-4e91-aef7-37d30fe8e3b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:50.194293 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d567b4-e5ff-4e91-aef7-37d30fe8e3b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:50.197502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d567b4-e5ff-4e91-aef7-37d30fe8e3b8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:50.202628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-062da08d-eb06-470e-8eff-cb43ab4dc063 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:50.202628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-062da08d-eb06-470e-8eff-cb43ab4dc063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:50.227631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 918/1842] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:50.236518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75f003-f387-41d5-9aee-e6b42fb1a206 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:50.236963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75f003-f387-41d5-9aee-e6b42fb1a206 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:50.237561 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75f003-f387-41d5-9aee-e6b42fb1a206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:50.284787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 925/1843] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:50 2022] POST /identity/v3/projects => generated 373 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:50.287970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d567b4-e5ff-4e91-aef7-37d30fe8e3b8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:50.290672 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0e75f003-f387-41d5-9aee-e6b42fb1a206 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:50.291198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-87d567b4-e5ff-4e91-aef7-37d30fe8e3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:50.291938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0e75f003-f387-41d5-9aee-e6b42fb1a206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:50.406304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:50.462314 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 919/1844] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:50.474202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75f003-f387-41d5-9aee-e6b42fb1a206 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:50.489564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d567b4-e5ff-4e91-aef7-37d30fe8e3b8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:50.490727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75f003-f387-41d5-9aee-e6b42fb1a206 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:50.496241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608eae1-23b4-4f3e-b78a-d42075a9e5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:50.497421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608eae1-23b4-4f3e-b78a-d42075a9e5de admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:50.497806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608eae1-23b4-4f3e-b78a-d42075a9e5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:50.498176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608eae1-23b4-4f3e-b78a-d42075a9e5de admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:50.499727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608eae1-23b4-4f3e-b78a-d42075a9e5de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:50.502101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608eae1-23b4-4f3e-b78a-d42075a9e5de admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:50.505533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367ff08-418e-4847-8329-15bbe9043f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:50.506256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367ff08-418e-4847-8329-15bbe9043f69 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:50.506608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367ff08-418e-4847-8329-15bbe9043f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:50.506877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367ff08-418e-4847-8329-15bbe9043f69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:50.507733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367ff08-418e-4847-8329-15bbe9043f69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:50.510281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367ff08-418e-4847-8329-15bbe9043f69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:50.513813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367ff08-418e-4847-8329-15bbe9043f69 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:50.542389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 920/1845] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:50.552033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 926/1846] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:50 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:50.552414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367ff08-418e-4847-8329-15bbe9043f69 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:50.553466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608eae1-23b4-4f3e-b78a-d42075a9e5de admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:50.569014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367ff08-418e-4847-8329-15bbe9043f69 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:50.574716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608eae1-23b4-4f3e-b78a-d42075a9e5de admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:50.575360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d937d2e-6e90-4cb9-a4cc-810e9e45d6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:50.576292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d937d2e-6e90-4cb9-a4cc-810e9e45d6a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:50.577060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d937d2e-6e90-4cb9-a4cc-810e9e45d6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:50.577434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d937d2e-6e90-4cb9-a4cc-810e9e45d6a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:50.578154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d937d2e-6e90-4cb9-a4cc-810e9e45d6a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:50.580664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d937d2e-6e90-4cb9-a4cc-810e9e45d6a7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:50.582622 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebb06b-1fc2-4098-ae0a-52b782668627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:50.583377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebb06b-1fc2-4098-ae0a-52b782668627 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:50.583750 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebb06b-1fc2-4098-ae0a-52b782668627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:50.584125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebb06b-1fc2-4098-ae0a-52b782668627 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:50.584874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebb06b-1fc2-4098-ae0a-52b782668627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:50.586640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebb06b-1fc2-4098-ae0a-52b782668627 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:50.613104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 927/1847] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:50.617640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebb06b-1fc2-4098-ae0a-52b782668627 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:50.637592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebb06b-1fc2-4098-ae0a-52b782668627 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:50.650927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a620d06-85f2-4a43-ab13-94c43be2f720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:50.652065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a620d06-85f2-4a43-ab13-94c43be2f720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:50.652450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a620d06-85f2-4a43-ab13-94c43be2f720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:50.653140 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a620d06-85f2-4a43-ab13-94c43be2f720 admin admin] PATH_INFO: `/v3/projects/87b84df7825c41d58c126797e9bab592/users/2605cb149e1343a491aef1d32aea97be/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:50.678953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 921/1848] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:50 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:50.681421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d937d2e-6e90-4cb9-a4cc-810e9e45d6a7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:50.702118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a620d06-85f2-4a43-ab13-94c43be2f720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b84df7825c41d58c126797e9bab592, user_id=2605cb149e1343a491aef1d32aea97be, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:50.706102 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a620d06-85f2-4a43-ab13-94c43be2f720 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:50.749509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 928/1849] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:50 2022] PUT /identity/v3/projects/87b84df7825c41d58c126797e9bab592/users/2605cb149e1343a491aef1d32aea97be/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:50.753072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a620d06-85f2-4a43-ab13-94c43be2f720 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:50.836517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d937d2e-6e90-4cb9-a4cc-810e9e45d6a7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:50.887881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a620d06-85f2-4a43-ab13-94c43be2f720 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:50.900741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a602e9a-25f5-49de-94b3-ce36d64fa07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:50.901370 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a602e9a-25f5-49de-94b3-ce36d64fa07a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:50.901644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a602e9a-25f5-49de-94b3-ce36d64fa07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:50.904494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a602e9a-25f5-49de-94b3-ce36d64fa07a admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:50.905240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a602e9a-25f5-49de-94b3-ce36d64fa07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:50.908429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a602e9a-25f5-49de-94b3-ce36d64fa07a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:50.939883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 929/1850] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:50.942313 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a602e9a-25f5-49de-94b3-ce36d64fa07a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:50.960797 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a602e9a-25f5-49de-94b3-ce36d64fa07a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:50.965805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227bb3e-3e09-464f-88ce-b74815491375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:50.966488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227bb3e-3e09-464f-88ce-b74815491375 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:50.966805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227bb3e-3e09-464f-88ce-b74815491375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:50.967078 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227bb3e-3e09-464f-88ce-b74815491375 admin admin] PATH_INFO: `/v3/projects/87b84df7825c41d58c126797e9bab592/users/2605cb149e1343a491aef1d32aea97be/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:50.969342 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227bb3e-3e09-464f-88ce-b74815491375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b84df7825c41d58c126797e9bab592, user_id=2605cb149e1343a491aef1d32aea97be, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:50.972379 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227bb3e-3e09-464f-88ce-b74815491375 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:50.997577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 930/1851] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:50 2022] PUT /identity/v3/projects/87b84df7825c41d58c126797e9bab592/users/2605cb149e1343a491aef1d32aea97be/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:50.999410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e25e3-0ca6-4a65-9a96-f3aa4f87bf7e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:51.000060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e25e3-0ca6-4a65-9a96-f3aa4f87bf7e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:51.000512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e25e3-0ca6-4a65-9a96-f3aa4f87bf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:51.016320 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f99ccd-5a8c-46b1-8ff8-0e9cd6ecbddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:51.017211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f99ccd-5a8c-46b1-8ff8-0e9cd6ecbddc admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:51.017653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f99ccd-5a8c-46b1-8ff8-0e9cd6ecbddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:51.018068 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f99ccd-5a8c-46b1-8ff8-0e9cd6ecbddc admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:51.018758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f99ccd-5a8c-46b1-8ff8-0e9cd6ecbddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:51.021843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f99ccd-5a8c-46b1-8ff8-0e9cd6ecbddc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:51.052830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6b7e25e3-0ca6-4a65-9a96-f3aa4f87bf7e None None] MFA Rules not processed for user `2605cb149e1343a491aef1d32aea97be`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:51.054642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6b7e25e3-0ca6-4a65-9a96-f3aa4f87bf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:51.071287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:51.103288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 922/1852] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:50 2022] POST /identity/v3/users => generated 293 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:51.109272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f99ccd-5a8c-46b1-8ff8-0e9cd6ecbddc admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:51.135339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f99ccd-5a8c-46b1-8ff8-0e9cd6ecbddc admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:51.238095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 931/1853] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:51.238095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6758103-43cf-4c88-b903-f67462d43531 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:51.238095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6758103-43cf-4c88-b903-f67462d43531 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:51.238095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6758103-43cf-4c88-b903-f67462d43531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:51.273179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d6758103-43cf-4c88-b903-f67462d43531 None None] MFA Rules not processed for user `5aac7f666e9345ca8f51fe7c7f9f74a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:51.274419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d6758103-43cf-4c88-b903-f67462d43531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:51.334021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:51.345333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46de12-5593-4dcc-9652-7b71b1b9a31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:51.346542 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46de12-5593-4dcc-9652-7b71b1b9a31d admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:51.347262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46de12-5593-4dcc-9652-7b71b1b9a31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:51.347955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46de12-5593-4dcc-9652-7b71b1b9a31d admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:51.348547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46de12-5593-4dcc-9652-7b71b1b9a31d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:51.351397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46de12-5593-4dcc-9652-7b71b1b9a31d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:51.392050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 923/1854] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:51.392050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46de12-5593-4dcc-9652-7b71b1b9a31d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:51.394421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ca46de12-5593-4dcc-9652-7b71b1b9a31d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:51.416415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 932/1855] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:51 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:51.417947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6758103-43cf-4c88-b903-f67462d43531 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:51.433837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6758103-43cf-4c88-b903-f67462d43531 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:51.440118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185814dd-e309-41c2-9a4f-6676287b6707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:51.440962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185814dd-e309-41c2-9a4f-6676287b6707 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:51.441437 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185814dd-e309-41c2-9a4f-6676287b6707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:51.441866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185814dd-e309-41c2-9a4f-6676287b6707 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:51.442718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185814dd-e309-41c2-9a4f-6676287b6707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:51.453810 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185814dd-e309-41c2-9a4f-6676287b6707 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:51.485882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 933/1856] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:51.490744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185814dd-e309-41c2-9a4f-6676287b6707 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:51.507444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185814dd-e309-41c2-9a4f-6676287b6707 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:51.533972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294efd86-0c65-4c33-8bcf-89cc0b0b3f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:51.534696 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294efd86-0c65-4c33-8bcf-89cc0b0b3f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:51.534962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294efd86-0c65-4c33-8bcf-89cc0b0b3f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:51.535589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294efd86-0c65-4c33-8bcf-89cc0b0b3f2a admin admin] PATH_INFO: `/v3/projects/b4f996b865f24db596b2d8abf3a19634/users/83d2e57ea7084aa1ac94ed1571ffb91c/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:51.538953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46de12-5593-4dcc-9652-7b71b1b9a31d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:51.544663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fc697-b4f8-4dce-82fb-304efb7b3816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:51.545434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fc697-b4f8-4dce-82fb-304efb7b3816 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:51.545710 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fc697-b4f8-4dce-82fb-304efb7b3816 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:51.546535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fc697-b4f8-4dce-82fb-304efb7b3816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:51.556402 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294efd86-0c65-4c33-8bcf-89cc0b0b3f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f996b865f24db596b2d8abf3a19634, user_id=83d2e57ea7084aa1ac94ed1571ffb91c, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:51.559114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294efd86-0c65-4c33-8bcf-89cc0b0b3f2a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:51.572175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fc697-b4f8-4dce-82fb-304efb7b3816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:51.574684 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fc697-b4f8-4dce-82fb-304efb7b3816 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:51.586415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 934/1857] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:51 2022] PUT /identity/v3/projects/b4f996b865f24db596b2d8abf3a19634/users/83d2e57ea7084aa1ac94ed1571ffb91c/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:51.588144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbea37-47b3-451e-8658-131946e52b70 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:51.588484 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbea37-47b3-451e-8658-131946e52b70 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:51.588757 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbea37-47b3-451e-8658-131946e52b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:51.593152 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 924/1858] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:51.624497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-18fbea37-47b3-451e-8658-131946e52b70 None None] MFA Rules not processed for user `83d2e57ea7084aa1ac94ed1571ffb91c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:51.625917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-18fbea37-47b3-451e-8658-131946e52b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:51.712941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:51.742592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 935/1859] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:51.748488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fc697-b4f8-4dce-82fb-304efb7b3816 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:51.760143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fc697-b4f8-4dce-82fb-304efb7b3816 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:51.843155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b574a5a-4ef5-451c-9707-486ddc7f5e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:51.843811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b574a5a-4ef5-451c-9707-486ddc7f5e4b admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:51.844119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b574a5a-4ef5-451c-9707-486ddc7f5e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:51.844392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b574a5a-4ef5-451c-9707-486ddc7f5e4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:51.845046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b574a5a-4ef5-451c-9707-486ddc7f5e4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:51.847028 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b574a5a-4ef5-451c-9707-486ddc7f5e4b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:51.881899 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 925/1860] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:51 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:51.885782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbea37-47b3-451e-8658-131946e52b70 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:51.897049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbea37-47b3-451e-8658-131946e52b70 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:51.959502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf8e08-dbdc-4d91-88f4-a7b14ce002e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:51.960129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf8e08-dbdc-4d91-88f4-a7b14ce002e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:51.960406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf8e08-dbdc-4d91-88f4-a7b14ce002e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:51.960702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf8e08-dbdc-4d91-88f4-a7b14ce002e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:51.961766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf8e08-dbdc-4d91-88f4-a7b14ce002e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:51.963285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf8e08-dbdc-4d91-88f4-a7b14ce002e0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:51.988161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 936/1861] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:51 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:51.992185 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b574a5a-4ef5-451c-9707-486ddc7f5e4b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:52.007253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b574a5a-4ef5-451c-9707-486ddc7f5e4b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:52.012169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f31ce-3a1d-44f8-95a0-73e3cdb5af6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:52.012761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f31ce-3a1d-44f8-95a0-73e3cdb5af6c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:52.013043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f31ce-3a1d-44f8-95a0-73e3cdb5af6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:52.013339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f31ce-3a1d-44f8-95a0-73e3cdb5af6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:52.013984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f31ce-3a1d-44f8-95a0-73e3cdb5af6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:52.015576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f31ce-3a1d-44f8-95a0-73e3cdb5af6c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:52.027176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 926/1862] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:52.033260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf8e08-dbdc-4d91-88f4-a7b14ce002e0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:52.061425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf8e08-dbdc-4d91-88f4-a7b14ce002e0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:52.066388 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e630c37-79b5-438d-996b-949bec1b220a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:52.067028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e630c37-79b5-438d-996b-949bec1b220a admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:52.067288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e630c37-79b5-438d-996b-949bec1b220a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:52.067562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e630c37-79b5-438d-996b-949bec1b220a admin admin] PATH_INFO: `/v3/projects/7d02823d1691441e9c7778e45f643266/users/88c6e0efbe7d46a195fae8ea632118f9/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:52.105391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e630c37-79b5-438d-996b-949bec1b220a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d02823d1691441e9c7778e45f643266, user_id=88c6e0efbe7d46a195fae8ea632118f9, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:52.108829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e630c37-79b5-438d-996b-949bec1b220a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:52.127022 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f31ce-3a1d-44f8-95a0-73e3cdb5af6c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:52.138335 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 937/1863] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:52 2022] PUT /identity/v3/projects/7d02823d1691441e9c7778e45f643266/users/88c6e0efbe7d46a195fae8ea632118f9/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:52.144989 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e630c37-79b5-438d-996b-949bec1b220a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:52.152227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f31ce-3a1d-44f8-95a0-73e3cdb5af6c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:52.237222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ee22b-bae3-40af-a1c0-bc688a422402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:52.238472 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ee22b-bae3-40af-a1c0-bc688a422402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:52.240250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ee22b-bae3-40af-a1c0-bc688a422402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:52.241065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ee22b-bae3-40af-a1c0-bc688a422402 admin admin] PATH_INFO: `/v3/users/ad7c3b2e18fd4edab09dc40efb02a377` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:52.243871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ee22b-bae3-40af-a1c0-bc688a422402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad7c3b2e18fd4edab09dc40efb02a377)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:52.246487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ee22b-bae3-40af-a1c0-bc688a422402 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:52.270075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:52.316158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e630c37-79b5-438d-996b-949bec1b220a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:52.322435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fd985-e85d-4378-9254-8d16f5182b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:52.323071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fd985-e85d-4378-9254-8d16f5182b8e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:52.323344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fd985-e85d-4378-9254-8d16f5182b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:52.323981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fd985-e85d-4378-9254-8d16f5182b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:52.324293 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fd985-e85d-4378-9254-8d16f5182b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:52.326126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fd985-e85d-4378-9254-8d16f5182b8e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:52.334199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-949ee22b-bae3-40af-a1c0-bc688a422402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad7c3b2e18fd4edab09dc40efb02a377'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:52.338425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 938/1864] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:52.339807 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd9573-4fa0-4e96-a5db-5e9c63f7fa2b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:52.340094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd9573-4fa0-4e96-a5db-5e9c63f7fa2b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:52.340367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd9573-4fa0-4e96-a5db-5e9c63f7fa2b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:52.341813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 939/1865] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:28:52.342452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd9573-4fa0-4e96-a5db-5e9c63f7fa2b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:52.356194 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-949ee22b-bae3-40af-a1c0-bc688a422402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad7c3b2e18fd4edab09dc40efb02a377'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:52.357122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-949ee22b-bae3-40af-a1c0-bc688a422402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad7c3b2e18fd4edab09dc40efb02a377'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:52.360647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd9573-4fa0-4e96-a5db-5e9c63f7fa2b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:52.366235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae357a4-fabd-433f-b973-d9792bbc99ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:52.367027 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae357a4-fabd-433f-b973-d9792bbc99ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:52.367359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae357a4-fabd-433f-b973-d9792bbc99ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:52.367821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae357a4-fabd-433f-b973-d9792bbc99ff admin admin] PATH_INFO: `/v3/projects/7d02823d1691441e9c7778e45f643266/users/88c6e0efbe7d46a195fae8ea632118f9/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:52.370984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae357a4-fabd-433f-b973-d9792bbc99ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d02823d1691441e9c7778e45f643266, user_id=88c6e0efbe7d46a195fae8ea632118f9, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:52.374737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae357a4-fabd-433f-b973-d9792bbc99ff admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:52.379231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-949ee22b-bae3-40af-a1c0-bc688a422402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad7c3b2e18fd4edab09dc40efb02a377'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:52.406967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 940/1866] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:52 2022] PUT /identity/v3/projects/7d02823d1691441e9c7778e45f643266/users/88c6e0efbe7d46a195fae8ea632118f9/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:52.414108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0189ebc-0a26-488a-9a40-fee232a8f31c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:52.414823 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0189ebc-0a26-488a-9a40-fee232a8f31c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:52.415248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0189ebc-0a26-488a-9a40-fee232a8f31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:52.415682 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-949ee22b-bae3-40af-a1c0-bc688a422402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad7c3b2e18fd4edab09dc40efb02a377'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:52.439072 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 927/1867] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:52 2022] DELETE /identity/v3/users/ad7c3b2e18fd4edab09dc40efb02a377 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:52.448097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce42d6-70e4-4ddb-aad0-bd270455fc95 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:52.449946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce42d6-70e4-4ddb-aad0-bd270455fc95 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:52.450342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce42d6-70e4-4ddb-aad0-bd270455fc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:52.462726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a0189ebc-0a26-488a-9a40-fee232a8f31c None None] MFA Rules not processed for user `e6d3b7887c13466ab8e3d9737f907c56`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:52.465047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a0189ebc-0a26-488a-9a40-fee232a8f31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:52.514716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:52.548543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b1ce42d6-70e4-4ddb-aad0-bd270455fc95 None None] MFA Rules not processed for user `88c6e0efbe7d46a195fae8ea632118f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:52.549918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b1ce42d6-70e4-4ddb-aad0-bd270455fc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:52.597573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 941/1868] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:28:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:52.599282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057c5a8-5e0b-46ef-b17c-8ac19ee36f49 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:52.599567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057c5a8-5e0b-46ef-b17c-8ac19ee36f49 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:52.599834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057c5a8-5e0b-46ef-b17c-8ac19ee36f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:52.664743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9057c5a8-5e0b-46ef-b17c-8ac19ee36f49 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:52.665933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9057c5a8-5e0b-46ef-b17c-8ac19ee36f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:52.757471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 928/1869] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:52.758766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce42d6-70e4-4ddb-aad0-bd270455fc95 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:52.765542 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b1ce42d6-70e4-4ddb-aad0-bd270455fc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:52.770252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 942/1870] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:52.774732 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057c5a8-5e0b-46ef-b17c-8ac19ee36f49 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:52.776892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9057c5a8-5e0b-46ef-b17c-8ac19ee36f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:52.792737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce42d6-70e4-4ddb-aad0-bd270455fc95 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:52.809405 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682e740-79f9-4fdb-a342-66132265270a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:52.810168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682e740-79f9-4fdb-a342-66132265270a service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:52.810517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682e740-79f9-4fdb-a342-66132265270a service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:52.812018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682e740-79f9-4fdb-a342-66132265270a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:52.836651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682e740-79f9-4fdb-a342-66132265270a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:52.842841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682e740-79f9-4fdb-a342-66132265270a service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:52.870338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 929/1871] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:52.871436 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682e740-79f9-4fdb-a342-66132265270a service placement] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:52.873017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a682e740-79f9-4fdb-a342-66132265270a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:52.970205 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057c5a8-5e0b-46ef-b17c-8ac19ee36f49 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:52.980200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdea9b-c87d-412b-8dc3-42634ed0d242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:52.981386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdea9b-c87d-412b-8dc3-42634ed0d242 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:52.981808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdea9b-c87d-412b-8dc3-42634ed0d242 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:52.982316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdea9b-c87d-412b-8dc3-42634ed0d242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:52.985919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-acbdea9b-c87d-412b-8dc3-42634ed0d242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:53.062414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682e740-79f9-4fdb-a342-66132265270a service placement] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:53.073038 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286d6c9-a1d9-418c-8a76-8e6bd88fe724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:53.074460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286d6c9-a1d9-418c-8a76-8e6bd88fe724 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:53.074907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286d6c9-a1d9-418c-8a76-8e6bd88fe724 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:53.075175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286d6c9-a1d9-418c-8a76-8e6bd88fe724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:53.079447 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f286d6c9-a1d9-418c-8a76-8e6bd88fe724 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:53.104575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdea9b-c87d-412b-8dc3-42634ed0d242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:53.106145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdea9b-c87d-412b-8dc3-42634ed0d242 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:53.117691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:53.121798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 943/1872] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:53.122903 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdea9b-c87d-412b-8dc3-42634ed0d242 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:53.124384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-acbdea9b-c87d-412b-8dc3-42634ed0d242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:53.173275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:53.183865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286d6c9-a1d9-418c-8a76-8e6bd88fe724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:53.185805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286d6c9-a1d9-418c-8a76-8e6bd88fe724 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:53.197145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 930/1873] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:53.235314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdea9b-c87d-412b-8dc3-42634ed0d242 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:53.240214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfefed-2ab8-4525-a65b-4856c5d95041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:53.241015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfefed-2ab8-4525-a65b-4856c5d95041 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:53.241472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfefed-2ab8-4525-a65b-4856c5d95041 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:53.241885 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfefed-2ab8-4525-a65b-4856c5d95041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:53.254772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfefed-2ab8-4525-a65b-4856c5d95041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:53.256452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfefed-2ab8-4525-a65b-4856c5d95041 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:53.279728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 944/1874] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:54.470059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286d6c9-a1d9-418c-8a76-8e6bd88fe724 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:54.472516 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f286d6c9-a1d9-418c-8a76-8e6bd88fe724 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:54.496878 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0e4e5-763e-4996-9501-dfc0394a69f3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:54.497528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0e4e5-763e-4996-9501-dfc0394a69f3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:54.497528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0e4e5-763e-4996-9501-dfc0394a69f3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:54.499052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 945/1875] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:28:54.502875 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286d6c9-a1d9-418c-8a76-8e6bd88fe724 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:54.520575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:54.521179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:54.521456 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:54.521728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] PATH_INFO: `/v3/projects/0fa33c59d868470aa3976580c77bfe40` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:54.523088 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fa33c59d868470aa3976580c77bfe40)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:54.524998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:54.541003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730639d9-8c43-4f39-b7d0-cd4ec09d78b2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:54.541335 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730639d9-8c43-4f39-b7d0-cd4ec09d78b2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:54.543131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730639d9-8c43-4f39-b7d0-cd4ec09d78b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:54.588241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-730639d9-8c43-4f39-b7d0-cd4ec09d78b2 None None] MFA Rules not processed for user `e6d3b7887c13466ab8e3d9737f907c56`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:54.588241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-730639d9-8c43-4f39-b7d0-cd4ec09d78b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:54.601922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 946/1876] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:28:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:54.648964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730639d9-8c43-4f39-b7d0-cd4ec09d78b2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:54.666765 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730639d9-8c43-4f39-b7d0-cd4ec09d78b2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:54.682220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fa33c59d868470aa3976580c77bfe40', 'request_id': 'req-d81218fb-8f09-4750-ae42-b43389167ba5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:54.706573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fa33c59d868470aa3976580c77bfe40', 'request_id': 'req-d81218fb-8f09-4750-ae42-b43389167ba5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:54.717157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fa33c59d868470aa3976580c77bfe40', 'request_id': 'req-d81218fb-8f09-4750-ae42-b43389167ba5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:54.743208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] The token cache is being invalidate because project 0fa33c59d868470aa3976580c77bfe40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:54.743208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:54.744272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:54.746676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 931/1877] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:54 2022] DELETE /identity/v3/projects/0fa33c59d868470aa3976580c77bfe40 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:54.756529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:54.761433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:54.800849 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f7ba0-d5f2-4d7c-9314-03c2bc3a0c8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:54.801536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f7ba0-d5f2-4d7c-9314-03c2bc3a0c8e service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:54.801811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f7ba0-d5f2-4d7c-9314-03c2bc3a0c8e service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:54.802718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f7ba0-d5f2-4d7c-9314-03c2bc3a0c8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:54.804837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0c3f7ba0-d5f2-4d7c-9314-03c2bc3a0c8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:54.819019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f7ba0-d5f2-4d7c-9314-03c2bc3a0c8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:54.821674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f7ba0-d5f2-4d7c-9314-03c2bc3a0c8e service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:54.835262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 947/1878] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:28:54.836243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f7ba0-d5f2-4d7c-9314-03c2bc3a0c8e service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:54.837770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0c3f7ba0-d5f2-4d7c-9314-03c2bc3a0c8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:54.929798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f7ba0-d5f2-4d7c-9314-03c2bc3a0c8e service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:54.936032 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:54.937044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:54.937416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:54.937754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] PATH_INFO: `/v3/users/4397170e783047629e56e48278019df1` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:54.939204 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4397170e783047629e56e48278019df1)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:54.944741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:54.954209 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81218fb-8f09-4750-ae42-b43389167ba5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:54.967885 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:54.968608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:54.968888 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:54.971495 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] PATH_INFO: `/v3/users/2605cb149e1343a491aef1d32aea97be` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:54.973960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2605cb149e1343a491aef1d32aea97be)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:54.980610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:55.018489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4397170e783047629e56e48278019df1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:55.037744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4397170e783047629e56e48278019df1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:55.039178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4397170e783047629e56e48278019df1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:55.045879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:55.075521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4397170e783047629e56e48278019df1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:55.095097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:55.118247 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2605cb149e1343a491aef1d32aea97be'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:55.124900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4397170e783047629e56e48278019df1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:55.130184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2605cb149e1343a491aef1d32aea97be'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:55.131016 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2605cb149e1343a491aef1d32aea97be'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:55.145677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2605cb149e1343a491aef1d32aea97be'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:55.150928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 948/1879] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:54 2022] DELETE /identity/v3/users/4397170e783047629e56e48278019df1 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:55.206442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2605cb149e1343a491aef1d32aea97be'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:55.224799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 932/1880] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:54 2022] DELETE /identity/v3/users/2605cb149e1343a491aef1d32aea97be => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:55.688321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:55.688321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:55.933867 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451adc9b-29c2-41fa-9636-79bac2073a07 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:56.023152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:56.023970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:56.024304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:56.024794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] PATH_INFO: `/v3/projects/2a1c1161a8874b5886c985009d1d4ff4` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:56.027421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a1c1161a8874b5886c985009d1d4ff4)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:56.029605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:56.110792 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a1c1161a8874b5886c985009d1d4ff4', 'request_id': 'req-173ad406-1173-44e5-bbc4-a8482d31e116'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.121398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a1c1161a8874b5886c985009d1d4ff4', 'request_id': 'req-173ad406-1173-44e5-bbc4-a8482d31e116'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.130392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a1c1161a8874b5886c985009d1d4ff4', 'request_id': 'req-173ad406-1173-44e5-bbc4-a8482d31e116'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.169227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] The token cache is being invalidate because project 2a1c1161a8874b5886c985009d1d4ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:56.169584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.173285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.176251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 949/1881] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:55 2022] DELETE /identity/v3/projects/2a1c1161a8874b5886c985009d1d4ff4 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:56.181335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:56.182916 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:56.206206 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5462324-6b01-4039-a456-5b9126510493 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:56.297544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:56.298959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:56.299311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:56.299663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] PATH_INFO: `/v3/users/5aac7f666e9345ca8f51fe7c7f9f74a8` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:56.301286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aac7f666e9345ca8f51fe7c7f9f74a8)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:56.308027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:56.394685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aac7f666e9345ca8f51fe7c7f9f74a8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.404548 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aac7f666e9345ca8f51fe7c7f9f74a8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.405287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aac7f666e9345ca8f51fe7c7f9f74a8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.412934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aac7f666e9345ca8f51fe7c7f9f74a8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.427359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aac7f666e9345ca8f51fe7c7f9f74a8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.438286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:56.439770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:56.440517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 933/1882] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:28:56 2022] DELETE /identity/v3/users/5aac7f666e9345ca8f51fe7c7f9f74a8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:56.459834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ad406-1173-44e5-bbc4-a8482d31e116 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:56.468935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:56.544762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec099128-730f-4d61-9f27-86fb0876ac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:56.545430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec099128-730f-4d61-9f27-86fb0876ac10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:56.545869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec099128-730f-4d61-9f27-86fb0876ac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:56.546279 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec099128-730f-4d61-9f27-86fb0876ac10 admin admin] PATH_INFO: `/v3/projects/87b84df7825c41d58c126797e9bab592` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:56.547638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec099128-730f-4d61-9f27-86fb0876ac10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87b84df7825c41d58c126797e9bab592)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:56.550122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec099128-730f-4d61-9f27-86fb0876ac10 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:56.624324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ec099128-730f-4d61-9f27-86fb0876ac10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87b84df7825c41d58c126797e9bab592', 'request_id': 'req-ec099128-730f-4d61-9f27-86fb0876ac10'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.636639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ec099128-730f-4d61-9f27-86fb0876ac10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87b84df7825c41d58c126797e9bab592', 'request_id': 'req-ec099128-730f-4d61-9f27-86fb0876ac10'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.643727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ec099128-730f-4d61-9f27-86fb0876ac10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87b84df7825c41d58c126797e9bab592', 'request_id': 'req-ec099128-730f-4d61-9f27-86fb0876ac10'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.650785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:56.652350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:56.656590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ec099128-730f-4d61-9f27-86fb0876ac10 admin admin] The token cache is being invalidate because project 87b84df7825c41d58c126797e9bab592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:56.657138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ec099128-730f-4d61-9f27-86fb0876ac10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.658150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ec099128-730f-4d61-9f27-86fb0876ac10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.665217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 950/1883] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:56 2022] DELETE /identity/v3/projects/87b84df7825c41d58c126797e9bab592 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:28:56.667619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186a8fc-00eb-4ee5-b7f2-bc20007bd774 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:56.667923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186a8fc-00eb-4ee5-b7f2-bc20007bd774 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:56.668302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186a8fc-00eb-4ee5-b7f2-bc20007bd774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:56.681600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b39230-e70d-4f60-843e-4eb2233f21f2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:56.695524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c186a8fc-00eb-4ee5-b7f2-bc20007bd774 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:56.696846 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c186a8fc-00eb-4ee5-b7f2-bc20007bd774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:56.716888 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:56.753096 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a3fb8-db0f-4f02-af23-e7384babcabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:56.753758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a3fb8-db0f-4f02-af23-e7384babcabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:56.754042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a3fb8-db0f-4f02-af23-e7384babcabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:56.754315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a3fb8-db0f-4f02-af23-e7384babcabf admin admin] PATH_INFO: `/v3/projects/5d757ddd957d44008050826243085995` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:56.755440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a3fb8-db0f-4f02-af23-e7384babcabf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d757ddd957d44008050826243085995)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:56.757430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a3fb8-db0f-4f02-af23-e7384babcabf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:56.770025 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 951/1884] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:56.778375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf33fe-4d54-4dc3-8cc1-01121b7601f8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:56.778902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf33fe-4d54-4dc3-8cc1-01121b7601f8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:56.779744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf33fe-4d54-4dc3-8cc1-01121b7601f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:56.803634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7fcf33fe-4d54-4dc3-8cc1-01121b7601f8 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:56.805122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7fcf33fe-4d54-4dc3-8cc1-01121b7601f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:56.814005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6f9a3fb8-db0f-4f02-af23-e7384babcabf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d757ddd957d44008050826243085995', 'request_id': 'req-6f9a3fb8-db0f-4f02-af23-e7384babcabf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.822843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6f9a3fb8-db0f-4f02-af23-e7384babcabf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d757ddd957d44008050826243085995', 'request_id': 'req-6f9a3fb8-db0f-4f02-af23-e7384babcabf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.831851 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6f9a3fb8-db0f-4f02-af23-e7384babcabf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d757ddd957d44008050826243085995', 'request_id': 'req-6f9a3fb8-db0f-4f02-af23-e7384babcabf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.843870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6f9a3fb8-db0f-4f02-af23-e7384babcabf admin admin] The token cache is being invalidate because project 5d757ddd957d44008050826243085995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:28:56.844228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6f9a3fb8-db0f-4f02-af23-e7384babcabf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.844895 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6f9a3fb8-db0f-4f02-af23-e7384babcabf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:28:56.846648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 934/1885] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:28:56 2022] DELETE /identity/v3/projects/5d757ddd957d44008050826243085995 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:28:56.850421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:56.851275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb1bbe-5b3e-4f0a-a65c-83dafb04d860 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:56.851631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb1bbe-5b3e-4f0a-a65c-83dafb04d860 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:56.852414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb1bbe-5b3e-4f0a-a65c-83dafb04d860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:56.878738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-15cb1bbe-5b3e-4f0a-a65c-83dafb04d860 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:56.879951 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-15cb1bbe-5b3e-4f0a-a65c-83dafb04d860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:56.885044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 952/1886] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:28:56.888425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 935/1887] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:56.890107 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf33fe-4d54-4dc3-8cc1-01121b7601f8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:56.895850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cebec-e6bc-4f74-b145-969d4dea440b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:56.896640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cebec-e6bc-4f74-b145-969d4dea440b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:56.896640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cebec-e6bc-4f74-b145-969d4dea440b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:56.901001 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf33fe-4d54-4dc3-8cc1-01121b7601f8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:56.906915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda4fb1-d298-405f-b721-2c773623f0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:56.907741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda4fb1-d298-405f-b721-2c773623f0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:56.907927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda4fb1-d298-405f-b721-2c773623f0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:56.908277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda4fb1-d298-405f-b721-2c773623f0d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:56.911898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda4fb1-d298-405f-b721-2c773623f0d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:56.911898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda4fb1-d298-405f-b721-2c773623f0d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:56.911898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda4fb1-d298-405f-b721-2c773623f0d3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:56.927419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 953/1888] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:56.928795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b81cebec-e6bc-4f74-b145-969d4dea440b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:56.930141 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b81cebec-e6bc-4f74-b145-969d4dea440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:56.932616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda4fb1-d298-405f-b721-2c773623f0d3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:56.940487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 936/1889] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:28:56.945321 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cebec-e6bc-4f74-b145-969d4dea440b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:56.946083 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda4fb1-d298-405f-b721-2c773623f0d3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:56.951199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc035b-b4cc-4402-8419-be0b4275a820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:56.951973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc035b-b4cc-4402-8419-be0b4275a820 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:56.952087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc035b-b4cc-4402-8419-be0b4275a820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:56.952340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc035b-b4cc-4402-8419-be0b4275a820 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:56.953150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc035b-b4cc-4402-8419-be0b4275a820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:56.955276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc035b-b4cc-4402-8419-be0b4275a820 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:56.955778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cebec-e6bc-4f74-b145-969d4dea440b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:56.963240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f35be-25a5-4ace-badc-78e3888835cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:56.963240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f35be-25a5-4ace-badc-78e3888835cb admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:56.963240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f35be-25a5-4ace-badc-78e3888835cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:56.963240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f35be-25a5-4ace-badc-78e3888835cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:56.963719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f35be-25a5-4ace-badc-78e3888835cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:28:56.963719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f35be-25a5-4ace-badc-78e3888835cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:56.965013 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f35be-25a5-4ace-badc-78e3888835cb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:56.989287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 937/1890] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:28:56.989287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f35be-25a5-4ace-badc-78e3888835cb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:56.995595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 954/1891] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:56 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:28:56.998981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc035b-b4cc-4402-8419-be0b4275a820 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:56.999652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f35be-25a5-4ace-badc-78e3888835cb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.060421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597376e-dd13-4618-8d8c-9ceb898c9787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.061025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597376e-dd13-4618-8d8c-9ceb898c9787 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.061353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597376e-dd13-4618-8d8c-9ceb898c9787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.061648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597376e-dd13-4618-8d8c-9ceb898c9787 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.062309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597376e-dd13-4618-8d8c-9ceb898c9787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.064268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597376e-dd13-4618-8d8c-9ceb898c9787 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.067302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc035b-b4cc-4402-8419-be0b4275a820 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.073561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df849c-f105-40f9-b167-a38e6e90942f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.074180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df849c-f105-40f9-b167-a38e6e90942f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.074478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df849c-f105-40f9-b167-a38e6e90942f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.074748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df849c-f105-40f9-b167-a38e6e90942f admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.075377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df849c-f105-40f9-b167-a38e6e90942f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.077881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df849c-f105-40f9-b167-a38e6e90942f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.097150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 938/1892] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:56 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:57.103338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 955/1893] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:56 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:57.103484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597376e-dd13-4618-8d8c-9ceb898c9787 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.104274 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df849c-f105-40f9-b167-a38e6e90942f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.113664 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df849c-f105-40f9-b167-a38e6e90942f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.122242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:57.168415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597376e-dd13-4618-8d8c-9ceb898c9787 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.171962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931f263-cf5e-4df8-ab15-bec2cebaab78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.172678 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931f263-cf5e-4df8-ab15-bec2cebaab78 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.172955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931f263-cf5e-4df8-ab15-bec2cebaab78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.173257 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931f263-cf5e-4df8-ab15-bec2cebaab78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.173703 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaedd9a-e69f-4f28-9347-1db204b4d213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.173902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931f263-cf5e-4df8-ab15-bec2cebaab78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.174315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaedd9a-e69f-4f28-9347-1db204b4d213 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.174592 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaedd9a-e69f-4f28-9347-1db204b4d213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.174992 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaedd9a-e69f-4f28-9347-1db204b4d213 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.175631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaedd9a-e69f-4f28-9347-1db204b4d213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.175741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931f263-cf5e-4df8-ab15-bec2cebaab78 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.177573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaedd9a-e69f-4f28-9347-1db204b4d213 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.186266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 956/1894] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:28:57.190723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931f263-cf5e-4df8-ab15-bec2cebaab78 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.204146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931f263-cf5e-4df8-ab15-bec2cebaab78 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.205772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 939/1895] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:57 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:28:57.209695 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaedd9a-e69f-4f28-9347-1db204b4d213 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.212270 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e9f6a-6d05-4378-9b86-42f84538c7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.212890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e9f6a-6d05-4378-9b86-42f84538c7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.213199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e9f6a-6d05-4378-9b86-42f84538c7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.213473 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e9f6a-6d05-4378-9b86-42f84538c7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.214128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e9f6a-6d05-4378-9b86-42f84538c7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.215743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e9f6a-6d05-4378-9b86-42f84538c7a2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.220364 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaedd9a-e69f-4f28-9347-1db204b4d213 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.225660 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11599a3e-6634-44ee-bd83-6fb4263980af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.226457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11599a3e-6634-44ee-bd83-6fb4263980af admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.226743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11599a3e-6634-44ee-bd83-6fb4263980af admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.227490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11599a3e-6634-44ee-bd83-6fb4263980af admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.228154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11599a3e-6634-44ee-bd83-6fb4263980af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.229859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11599a3e-6634-44ee-bd83-6fb4263980af admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.234935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 957/1896] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:57.235470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e9f6a-6d05-4378-9b86-42f84538c7a2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.243145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 940/1897] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:28:57.250176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11599a3e-6634-44ee-bd83-6fb4263980af admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.250617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e9f6a-6d05-4378-9b86-42f84538c7a2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.255473 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24edc57-66ed-4eda-8b05-6868f20249c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.256144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24edc57-66ed-4eda-8b05-6868f20249c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.256419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24edc57-66ed-4eda-8b05-6868f20249c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.256686 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24edc57-66ed-4eda-8b05-6868f20249c8 admin admin] PATH_INFO: `/v3/projects/518ea7ac8ba9429688cceb5d40067fb5/users/e6e77c183c7d459fb8042d4455477f7f/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.261471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11599a3e-6634-44ee-bd83-6fb4263980af admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.267057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed8b83-37c1-4869-81c4-f40809cda38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.267673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed8b83-37c1-4869-81c4-f40809cda38c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.267954 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed8b83-37c1-4869-81c4-f40809cda38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.268238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed8b83-37c1-4869-81c4-f40809cda38c admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.268868 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed8b83-37c1-4869-81c4-f40809cda38c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.271485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed8b83-37c1-4869-81c4-f40809cda38c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.274460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24edc57-66ed-4eda-8b05-6868f20249c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=518ea7ac8ba9429688cceb5d40067fb5, user_id=e6e77c183c7d459fb8042d4455477f7f, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.277287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24edc57-66ed-4eda-8b05-6868f20249c8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.279647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:57.286496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 941/1898] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:57.291018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed8b83-37c1-4869-81c4-f40809cda38c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.301754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed8b83-37c1-4869-81c4-f40809cda38c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.310082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 958/1899] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:57 2022] PUT /identity/v3/projects/518ea7ac8ba9429688cceb5d40067fb5/users/e6e77c183c7d459fb8042d4455477f7f/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:57.314993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982fd4e-7216-42a6-af49-b83952e8e83f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.315500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982fd4e-7216-42a6-af49-b83952e8e83f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.315617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982fd4e-7216-42a6-af49-b83952e8e83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.335753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1982fd4e-7216-42a6-af49-b83952e8e83f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:57.337073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1982fd4e-7216-42a6-af49-b83952e8e83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:57.387945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e0c00-725d-4c81-89fd-4a323de3b98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.389137 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e0c00-725d-4c81-89fd-4a323de3b98c admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.389422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e0c00-725d-4c81-89fd-4a323de3b98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.389702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e0c00-725d-4c81-89fd-4a323de3b98c admin admin] PATH_INFO: `/v3/projects/959d8cd72ac744628842d835f315c75d/users/5cd4c6d9f4a746f982506e2898268ad5/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.403670 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 959/1900] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:28:57.406290 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e0c00-725d-4c81-89fd-4a323de3b98c admin admin] RBAC: Authorizing `identity:create_grant(project_id=959d8cd72ac744628842d835f315c75d, user_id=5cd4c6d9f4a746f982506e2898268ad5, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.409040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e0c00-725d-4c81-89fd-4a323de3b98c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.414202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982fd4e-7216-42a6-af49-b83952e8e83f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.416079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1982fd4e-7216-42a6-af49-b83952e8e83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:57.426312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 942/1901] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:57 2022] PUT /identity/v3/projects/959d8cd72ac744628842d835f315c75d/users/5cd4c6d9f4a746f982506e2898268ad5/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:57.438765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0688e-6f02-460f-a10f-eb1b5e2a25da None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.439301 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0688e-6f02-460f-a10f-eb1b5e2a25da None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.439657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0688e-6f02-460f-a10f-eb1b5e2a25da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.471168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-afd0688e-6f02-460f-a10f-eb1b5e2a25da None None] MFA Rules not processed for user `5cd4c6d9f4a746f982506e2898268ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:57.473021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-afd0688e-6f02-460f-a10f-eb1b5e2a25da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:57.475886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982fd4e-7216-42a6-af49-b83952e8e83f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.528831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f0a2f-c156-4979-a8a1-161a6dd06f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.529614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f0a2f-c156-4979-a8a1-161a6dd06f55 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.529972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f0a2f-c156-4979-a8a1-161a6dd06f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.530317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f0a2f-c156-4979-a8a1-161a6dd06f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.534704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:57.541368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 943/1902] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:57.545016 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0688e-6f02-460f-a10f-eb1b5e2a25da None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.591808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f0a2f-c156-4979-a8a1-161a6dd06f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.593201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f0a2f-c156-4979-a8a1-161a6dd06f55 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.598677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:57.604616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 960/1903] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:28:57.616665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0688e-6f02-460f-a10f-eb1b5e2a25da None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.621259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaff16-12b9-4364-9802-4d34124d4fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.621898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaff16-12b9-4364-9802-4d34124d4fbf admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.622199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaff16-12b9-4364-9802-4d34124d4fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.622550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaff16-12b9-4364-9802-4d34124d4fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.623180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaff16-12b9-4364-9802-4d34124d4fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.625135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaff16-12b9-4364-9802-4d34124d4fbf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.653573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 944/1904] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:57 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:57.658064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f0a2f-c156-4979-a8a1-161a6dd06f55 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.669938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f0a2f-c156-4979-a8a1-161a6dd06f55 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.729976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b34c9d-7f75-4c24-936e-c356ddfb0a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.730654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b34c9d-7f75-4c24-936e-c356ddfb0a71 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.730861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b34c9d-7f75-4c24-936e-c356ddfb0a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.731129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b34c9d-7f75-4c24-936e-c356ddfb0a71 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.731796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b34c9d-7f75-4c24-936e-c356ddfb0a71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.733924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b34c9d-7f75-4c24-936e-c356ddfb0a71 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.756814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 961/1905] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:57 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:28:57.761008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaff16-12b9-4364-9802-4d34124d4fbf admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.771827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaff16-12b9-4364-9802-4d34124d4fbf admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.776783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83356191-4d95-457c-aebf-e23664329cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.777580 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83356191-4d95-457c-aebf-e23664329cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.777918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83356191-4d95-457c-aebf-e23664329cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.778260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83356191-4d95-457c-aebf-e23664329cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.778976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83356191-4d95-457c-aebf-e23664329cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.781183 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83356191-4d95-457c-aebf-e23664329cd8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.791879 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 945/1906] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:57.796300 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b34c9d-7f75-4c24-936e-c356ddfb0a71 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.806614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b34c9d-7f75-4c24-936e-c356ddfb0a71 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.811423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8b5f8-ceee-4aca-8758-bfeeb5b634cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.812086 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8b5f8-ceee-4aca-8758-bfeeb5b634cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.812365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8b5f8-ceee-4aca-8758-bfeeb5b634cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.812636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8b5f8-ceee-4aca-8758-bfeeb5b634cb admin admin] PATH_INFO: `/v3/projects/ff018089532145a3aadc4bdc0f480ae1/users/19287ee340be4e3aa62645fc4eb4a81e/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.827987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8b5f8-ceee-4aca-8758-bfeeb5b634cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff018089532145a3aadc4bdc0f480ae1, user_id=19287ee340be4e3aa62645fc4eb4a81e, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.830712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8b5f8-ceee-4aca-8758-bfeeb5b634cb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.845584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 962/1907] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:57 2022] PUT /identity/v3/projects/ff018089532145a3aadc4bdc0f480ae1/users/19287ee340be4e3aa62645fc4eb4a81e/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:57.849767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83356191-4d95-457c-aebf-e23664329cd8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.860468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83356191-4d95-457c-aebf-e23664329cd8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.923178 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9a340-4d26-4b7f-aa1b-c3bdbd1cada8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.923627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9a340-4d26-4b7f-aa1b-c3bdbd1cada8 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.923894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9a340-4d26-4b7f-aa1b-c3bdbd1cada8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.924167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9a340-4d26-4b7f-aa1b-c3bdbd1cada8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.924832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9a340-4d26-4b7f-aa1b-c3bdbd1cada8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.926649 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9a340-4d26-4b7f-aa1b-c3bdbd1cada8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.941121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 946/1908] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:57.945421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8b5f8-ceee-4aca-8758-bfeeb5b634cb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:57.955794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8b5f8-ceee-4aca-8758-bfeeb5b634cb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:57.965863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e3e88-3d56-4337-a0f2-39c8e8344aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:57.966715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e3e88-3d56-4337-a0f2-39c8e8344aac admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.967085 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e3e88-3d56-4337-a0f2-39c8e8344aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.967428 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e3e88-3d56-4337-a0f2-39c8e8344aac admin admin] PATH_INFO: `/v3/projects/ff018089532145a3aadc4bdc0f480ae1/users/19287ee340be4e3aa62645fc4eb4a81e/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:57.969770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e3e88-3d56-4337-a0f2-39c8e8344aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff018089532145a3aadc4bdc0f480ae1, user_id=19287ee340be4e3aa62645fc4eb4a81e, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:57.972964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e3e88-3d56-4337-a0f2-39c8e8344aac admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:57.984626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 963/1909] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:57 2022] PUT /identity/v3/projects/ff018089532145a3aadc4bdc0f480ae1/users/19287ee340be4e3aa62645fc4eb4a81e/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:57.997393 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166ec6f-5c87-4fbe-9e89-4b73abec6a57 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:57.997770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166ec6f-5c87-4fbe-9e89-4b73abec6a57 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:57.998120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166ec6f-5c87-4fbe-9e89-4b73abec6a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:58.040183 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f166ec6f-5c87-4fbe-9e89-4b73abec6a57 None None] MFA Rules not processed for user `19287ee340be4e3aa62645fc4eb4a81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:58.041505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f166ec6f-5c87-4fbe-9e89-4b73abec6a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:58.102907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:58.114371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 947/1910] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:58.125941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e3e88-3d56-4337-a0f2-39c8e8344aac admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:58.127973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6f5e3e88-3d56-4337-a0f2-39c8e8344aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:58.187523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e3e88-3d56-4337-a0f2-39c8e8344aac admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:58.191615 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3bff0a-d1e3-4ff7-86fe-c1fca791c9d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:58.192233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bff0a-d1e3-4ff7-86fe-c1fca791c9d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:58.192511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bff0a-d1e3-4ff7-86fe-c1fca791c9d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:58.192784 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bff0a-d1e3-4ff7-86fe-c1fca791c9d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:58.202043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3bff0a-d1e3-4ff7-86fe-c1fca791c9d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:58.203515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3bff0a-d1e3-4ff7-86fe-c1fca791c9d3 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:58.213475 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 964/1911] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:58.294344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166ec6f-5c87-4fbe-9e89-4b73abec6a57 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:58.303948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166ec6f-5c87-4fbe-9e89-4b73abec6a57 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:58.317928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe208054-b37a-456a-8f09-5632f6dda53d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:58.318581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe208054-b37a-456a-8f09-5632f6dda53d service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:58.318851 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe208054-b37a-456a-8f09-5632f6dda53d service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:58.319143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe208054-b37a-456a-8f09-5632f6dda53d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:58.375044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d7f52-e995-45f6-be79-53cd2d234847 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:58.375624 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d7f52-e995-45f6-be79-53cd2d234847 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:58.375936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d7f52-e995-45f6-be79-53cd2d234847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:58.417470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-561d7f52-e995-45f6-be79-53cd2d234847 None None] MFA Rules not processed for user `e6e77c183c7d459fb8042d4455477f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:58.419970 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-561d7f52-e995-45f6-be79-53cd2d234847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:58.424242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe208054-b37a-456a-8f09-5632f6dda53d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:58.426087 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe208054-b37a-456a-8f09-5632f6dda53d service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:58.437423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 948/1912] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:28:58.483961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:58.516360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 965/1913] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:58.521441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe208054-b37a-456a-8f09-5632f6dda53d service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:58.624537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe208054-b37a-456a-8f09-5632f6dda53d service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:58.630629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623aa6f0-dbae-44aa-8ea0-ae98a0805437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:58.633213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623aa6f0-dbae-44aa-8ea0-ae98a0805437 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:58.633923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623aa6f0-dbae-44aa-8ea0-ae98a0805437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:58.634519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623aa6f0-dbae-44aa-8ea0-ae98a0805437 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:58.635519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623aa6f0-dbae-44aa-8ea0-ae98a0805437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:58.637931 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623aa6f0-dbae-44aa-8ea0-ae98a0805437 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:58.680517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 949/1914] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:28:58 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:28:58.684283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d7f52-e995-45f6-be79-53cd2d234847 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:58.695891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d7f52-e995-45f6-be79-53cd2d234847 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:58.762491 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a07849-29f6-44e2-b0e5-04d27aaf1b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:58.763156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a07849-29f6-44e2-b0e5-04d27aaf1b85 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:58.763381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a07849-29f6-44e2-b0e5-04d27aaf1b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:58.763650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a07849-29f6-44e2-b0e5-04d27aaf1b85 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:58.764286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a07849-29f6-44e2-b0e5-04d27aaf1b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:58.766299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a07849-29f6-44e2-b0e5-04d27aaf1b85 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:58.792915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 966/1915] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:28:58 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:28:58.794310 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623aa6f0-dbae-44aa-8ea0-ae98a0805437 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:58.804879 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623aa6f0-dbae-44aa-8ea0-ae98a0805437 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:58.810196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee46f8-3b56-432a-bc05-3e81efbd2701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:58.810798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee46f8-3b56-432a-bc05-3e81efbd2701 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:58.811050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee46f8-3b56-432a-bc05-3e81efbd2701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:58.811349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee46f8-3b56-432a-bc05-3e81efbd2701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:58.811999 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee46f8-3b56-432a-bc05-3e81efbd2701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:58.813699 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee46f8-3b56-432a-bc05-3e81efbd2701 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:58.824012 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 950/1916] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:58.828200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a07849-29f6-44e2-b0e5-04d27aaf1b85 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:58.838286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a07849-29f6-44e2-b0e5-04d27aaf1b85 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:58.842923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cffc5-06c4-4c62-b80c-908271a003db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:58.843573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cffc5-06c4-4c62-b80c-908271a003db admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:58.843850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cffc5-06c4-4c62-b80c-908271a003db admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:58.844181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cffc5-06c4-4c62-b80c-908271a003db admin admin] PATH_INFO: `/v3/projects/84e45a668f214e35be48d9782e5a4042/users/00cee67e83df4307809596fb2af2abee/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:58.859442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cffc5-06c4-4c62-b80c-908271a003db admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e45a668f214e35be48d9782e5a4042, user_id=00cee67e83df4307809596fb2af2abee, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:58.862395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cffc5-06c4-4c62-b80c-908271a003db admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:58.876821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 967/1917] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:58 2022] PUT /identity/v3/projects/84e45a668f214e35be48d9782e5a4042/users/00cee67e83df4307809596fb2af2abee/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:28:58.880877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee46f8-3b56-432a-bc05-3e81efbd2701 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:58.890889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee46f8-3b56-432a-bc05-3e81efbd2701 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:58.947934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c3f6b-e2dc-4dc2-9e6f-f50c4388a9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:58.948637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c3f6b-e2dc-4dc2-9e6f-f50c4388a9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:58.948936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c3f6b-e2dc-4dc2-9e6f-f50c4388a9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:58.949214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c3f6b-e2dc-4dc2-9e6f-f50c4388a9e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:58.949853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c3f6b-e2dc-4dc2-9e6f-f50c4388a9e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:58.951475 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c3f6b-e2dc-4dc2-9e6f-f50c4388a9e1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:58.961319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 951/1918] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:28:58.965890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cffc5-06c4-4c62-b80c-908271a003db admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:58.977285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cffc5-06c4-4c62-b80c-908271a003db admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:28:58.982134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55decf49-66a0-4e10-bb1f-97e2565a8ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:28:58.982777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55decf49-66a0-4e10-bb1f-97e2565a8ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:58.983057 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55decf49-66a0-4e10-bb1f-97e2565a8ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:58.983316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55decf49-66a0-4e10-bb1f-97e2565a8ee4 admin admin] PATH_INFO: `/v3/projects/84e45a668f214e35be48d9782e5a4042/users/00cee67e83df4307809596fb2af2abee/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:58.985290 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55decf49-66a0-4e10-bb1f-97e2565a8ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e45a668f214e35be48d9782e5a4042, user_id=00cee67e83df4307809596fb2af2abee, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:28:58.987891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55decf49-66a0-4e10-bb1f-97e2565a8ee4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:28:58.998238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 968/1919] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:28:58 2022] PUT /identity/v3/projects/84e45a668f214e35be48d9782e5a4042/users/00cee67e83df4307809596fb2af2abee/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:28:59.762618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0bb5-c851-4b43-8b29-2e9b7d4a7ae8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:28:59.763053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0bb5-c851-4b43-8b29-2e9b7d4a7ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:28:59.763286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0bb5-c851-4b43-8b29-2e9b7d4a7ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:28:59.771726 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:59.798047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-690b0bb5-c851-4b43-8b29-2e9b7d4a7ae8 None None] MFA Rules not processed for user `00cee67e83df4307809596fb2af2abee`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:28:59.799390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-690b0bb5-c851-4b43-8b29-2e9b7d4a7ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:28:59.921606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 952/1920] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:28:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:28:59.935991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55decf49-66a0-4e10-bb1f-97e2565a8ee4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:28:59.996452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55decf49-66a0-4e10-bb1f-97e2565a8ee4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:00.000100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb4d08-2fe0-4397-a686-8808cc8a0648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:00.000777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb4d08-2fe0-4397-a686-8808cc8a0648 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:00.001061 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb4d08-2fe0-4397-a686-8808cc8a0648 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:00.001376 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb4d08-2fe0-4397-a686-8808cc8a0648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:00.050544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:00.059598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb4d08-2fe0-4397-a686-8808cc8a0648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:00.061358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb4d08-2fe0-4397-a686-8808cc8a0648 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:00.070830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 969/1921] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:28:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:02.401761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0bb5-c851-4b43-8b29-2e9b7d4a7ae8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:02.403348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-690b0bb5-c851-4b43-8b29-2e9b7d4a7ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:02.484452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0bb5-c851-4b43-8b29-2e9b7d4a7ae8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:02.488935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:02.489608 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:02.489884 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:02.490154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] PATH_INFO: `/v3/users/f2fce365cc8b42a4877957d48771cd1f` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:02.491337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2fce365cc8b42a4877957d48771cd1f)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:02.493263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:02.507573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb4d08-2fe0-4397-a686-8808cc8a0648 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:02.509775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e2fb4d08-2fe0-4397-a686-8808cc8a0648 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:02.523251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb4d08-2fe0-4397-a686-8808cc8a0648 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:02.527597 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579d7e7-46a3-4565-8ddb-d0ea56925826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:02.528213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579d7e7-46a3-4565-8ddb-d0ea56925826 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:02.528488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579d7e7-46a3-4565-8ddb-d0ea56925826 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:02.528757 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579d7e7-46a3-4565-8ddb-d0ea56925826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:02.537915 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579d7e7-46a3-4565-8ddb-d0ea56925826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:02.539342 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579d7e7-46a3-4565-8ddb-d0ea56925826 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:02.540343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2fce365cc8b42a4877957d48771cd1f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:02.549219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 970/1922] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:29:02.551343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2fce365cc8b42a4877957d48771cd1f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:02.551970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2fce365cc8b42a4877957d48771cd1f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:02.558958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2fce365cc8b42a4877957d48771cd1f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:02.572265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2fce365cc8b42a4877957d48771cd1f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:02.578404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 953/1923] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:02 2022] DELETE /identity/v3/users/f2fce365cc8b42a4877957d48771cd1f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:02.582936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653336db-685b-462a-b321-7d5914882733 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:02.583224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653336db-685b-462a-b321-7d5914882733 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:02.583497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653336db-685b-462a-b321-7d5914882733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:02.604498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-653336db-685b-462a-b321-7d5914882733 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:02.605721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-653336db-685b-462a-b321-7d5914882733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:02.669736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 971/1924] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:02.677298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:02.678798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:02.689851 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:02.739878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1d80a-b6cf-4467-be57-f77dffe7e20d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:02.743692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4e78d-00cd-43ac-8b94-9f0217bcb44b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:02.744349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4e78d-00cd-43ac-8b94-9f0217bcb44b service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:02.744607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4e78d-00cd-43ac-8b94-9f0217bcb44b service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:02.744916 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4e78d-00cd-43ac-8b94-9f0217bcb44b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:02.755415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4e78d-00cd-43ac-8b94-9f0217bcb44b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:02.756919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4e78d-00cd-43ac-8b94-9f0217bcb44b service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:02.766787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 954/1925] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:03.025658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653336db-685b-462a-b321-7d5914882733 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:03.027398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-653336db-685b-462a-b321-7d5914882733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:03.046909 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4e78d-00cd-43ac-8b94-9f0217bcb44b service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:03.048692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-57b4e78d-00cd-43ac-8b94-9f0217bcb44b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:03.075518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4e78d-00cd-43ac-8b94-9f0217bcb44b service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:03.080967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:03.081701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:03.082044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:03.082377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] PATH_INFO: `/v3/projects/6f8e5da07136490988f2be09153304dc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:03.083599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f8e5da07136490988f2be09153304dc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:03.085616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:03.090606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653336db-685b-462a-b321-7d5914882733 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:03.095543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668ae67-4420-42e1-af55-275867e3f9dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:03.096713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668ae67-4420-42e1-af55-275867e3f9dd service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:03.097757 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668ae67-4420-42e1-af55-275867e3f9dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:03.098413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668ae67-4420-42e1-af55-275867e3f9dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:03.100279 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7668ae67-4420-42e1-af55-275867e3f9dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:03.145519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f8e5da07136490988f2be09153304dc', 'request_id': 'req-546be9a8-c0f1-46eb-aab4-8c30965a38a7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:03.154856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f8e5da07136490988f2be09153304dc', 'request_id': 'req-546be9a8-c0f1-46eb-aab4-8c30965a38a7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:03.161993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f8e5da07136490988f2be09153304dc', 'request_id': 'req-546be9a8-c0f1-46eb-aab4-8c30965a38a7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:03.170764 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668ae67-4420-42e1-af55-275867e3f9dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:03.173288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668ae67-4420-42e1-af55-275867e3f9dd service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:03.173968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] The token cache is being invalidate because project 6f8e5da07136490988f2be09153304dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:03.174262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:03.174919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:03.176595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 955/1926] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:03 2022] DELETE /identity/v3/projects/6f8e5da07136490988f2be09153304dc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:29:03.180648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:03.182438 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:03.184098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 972/1927] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:03.258552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546be9a8-c0f1-46eb-aab4-8c30965a38a7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:03.264951 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1eb79a-c374-40ce-ab2b-c692901e41ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:03.265533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1eb79a-c374-40ce-ab2b-c692901e41ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:03.265862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1eb79a-c374-40ce-ab2b-c692901e41ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:03.266140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1eb79a-c374-40ce-ab2b-c692901e41ef admin admin] PATH_INFO: `/v3/users/c05aa65bf77e48818a0340232d4e72f7` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:03.267650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1eb79a-c374-40ce-ab2b-c692901e41ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=c05aa65bf77e48818a0340232d4e72f7)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:03.269949 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1eb79a-c374-40ce-ab2b-c692901e41ef admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:03.338285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:03.352189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1a1eb79a-c374-40ce-ab2b-c692901e41ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c05aa65bf77e48818a0340232d4e72f7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:03.364878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1a1eb79a-c374-40ce-ab2b-c692901e41ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c05aa65bf77e48818a0340232d4e72f7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:03.365832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1a1eb79a-c374-40ce-ab2b-c692901e41ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c05aa65bf77e48818a0340232d4e72f7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:03.392828 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1a1eb79a-c374-40ce-ab2b-c692901e41ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c05aa65bf77e48818a0340232d4e72f7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:03.412353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1a1eb79a-c374-40ce-ab2b-c692901e41ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c05aa65bf77e48818a0340232d4e72f7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:03.441313 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 956/1928] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:03 2022] DELETE /identity/v3/users/c05aa65bf77e48818a0340232d4e72f7 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:04.057504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668ae67-4420-42e1-af55-275867e3f9dd service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:04.070005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668ae67-4420-42e1-af55-275867e3f9dd service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:04.102232 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:04.132690 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:04.133362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:04.133627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:04.133893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af admin admin] PATH_INFO: `/v3/projects/680a6904139b4a68967d055bba9f9447` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:04.135133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af admin admin] RBAC: Authorizing `identity:delete_project(project_id=680a6904139b4a68967d055bba9f9447)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:04.137104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:04.184013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '680a6904139b4a68967d055bba9f9447', 'request_id': 'req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:04.191965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '680a6904139b4a68967d055bba9f9447', 'request_id': 'req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:04.197410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '680a6904139b4a68967d055bba9f9447', 'request_id': 'req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:04.207928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af admin admin] The token cache is being invalidate because project 680a6904139b4a68967d055bba9f9447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:04.208213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:04.208827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-62d7cdb8-63cc-48fc-8b27-1d3b265c99af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:04.210305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 973/1929] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:04 2022] DELETE /identity/v3/projects/680a6904139b4a68967d055bba9f9447 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:04.216484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fe2bb-696d-4dec-b1c7-7a2f002cc26b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:04.216776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fe2bb-696d-4dec-b1c7-7a2f002cc26b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:04.217046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fe2bb-696d-4dec-b1c7-7a2f002cc26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:04.239060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d77fe2bb-696d-4dec-b1c7-7a2f002cc26b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:04.240323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d77fe2bb-696d-4dec-b1c7-7a2f002cc26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:04.308898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 957/1930] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:04.328494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa449c-2bc5-4ebc-909f-ca67c70b1562 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:04.328692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa449c-2bc5-4ebc-909f-ca67c70b1562 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:04.328894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa449c-2bc5-4ebc-909f-ca67c70b1562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:04.352167 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-57fa449c-2bc5-4ebc-909f-ca67c70b1562 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:04.353426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-57fa449c-2bc5-4ebc-909f-ca67c70b1562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:04.365419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 974/1931] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:04.370252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fe2bb-696d-4dec-b1c7-7a2f002cc26b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:04.380128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fe2bb-696d-4dec-b1c7-7a2f002cc26b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:04.385263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea805a12-d76a-480c-97b6-4c7bcc529034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:04.385955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea805a12-d76a-480c-97b6-4c7bcc529034 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:04.386288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea805a12-d76a-480c-97b6-4c7bcc529034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:04.386560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea805a12-d76a-480c-97b6-4c7bcc529034 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:04.387249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea805a12-d76a-480c-97b6-4c7bcc529034 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:29:04.387550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea805a12-d76a-480c-97b6-4c7bcc529034 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:04.389162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea805a12-d76a-480c-97b6-4c7bcc529034 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:04.403320 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 958/1932] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:29:04.408160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa449c-2bc5-4ebc-909f-ca67c70b1562 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:04.418051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa449c-2bc5-4ebc-909f-ca67c70b1562 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:04.423551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c9175-249c-4ee5-8462-b8372642028a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:04.424161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c9175-249c-4ee5-8462-b8372642028a admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:04.424441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c9175-249c-4ee5-8462-b8372642028a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:04.424714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c9175-249c-4ee5-8462-b8372642028a admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:04.425392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c9175-249c-4ee5-8462-b8372642028a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:04.427302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c9175-249c-4ee5-8462-b8372642028a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:04.459962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 975/1933] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:04 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:04.464493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea805a12-d76a-480c-97b6-4c7bcc529034 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:04.474852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea805a12-d76a-480c-97b6-4c7bcc529034 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:04.531160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ce604-b068-43c7-95bb-cc128eca18b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:04.531784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ce604-b068-43c7-95bb-cc128eca18b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:04.532090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ce604-b068-43c7-95bb-cc128eca18b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:04.532357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ce604-b068-43c7-95bb-cc128eca18b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:04.532982 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ce604-b068-43c7-95bb-cc128eca18b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:04.534996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ce604-b068-43c7-95bb-cc128eca18b1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:04.560172 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 959/1934] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:04 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:29:04.561485 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c9175-249c-4ee5-8462-b8372642028a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:04.571691 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c9175-249c-4ee5-8462-b8372642028a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:04.576515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e932-1dea-4934-aba2-0e1e4ed8e176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:04.577188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e932-1dea-4934-aba2-0e1e4ed8e176 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:04.577461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e932-1dea-4934-aba2-0e1e4ed8e176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:04.577732 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e932-1dea-4934-aba2-0e1e4ed8e176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:04.578383 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a68e932-1dea-4934-aba2-0e1e4ed8e176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:04.580175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a68e932-1dea-4934-aba2-0e1e4ed8e176 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:04.590807 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 976/1935] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:04.594779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ce604-b068-43c7-95bb-cc128eca18b1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:04.605715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ce604-b068-43c7-95bb-cc128eca18b1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:04.610330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee67a4-a256-4800-b191-defb337fd9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:04.610933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee67a4-a256-4800-b191-defb337fd9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:04.611209 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee67a4-a256-4800-b191-defb337fd9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:04.611477 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee67a4-a256-4800-b191-defb337fd9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:04.612126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee67a4-a256-4800-b191-defb337fd9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:04.613762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee67a4-a256-4800-b191-defb337fd9a8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:04.624236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 960/1936] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:04.628503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e932-1dea-4934-aba2-0e1e4ed8e176 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:04.638598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e932-1dea-4934-aba2-0e1e4ed8e176 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:04.643596 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470f9f7-57df-41bb-a9d9-967464eab9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:04.644248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470f9f7-57df-41bb-a9d9-967464eab9cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:04.644529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470f9f7-57df-41bb-a9d9-967464eab9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:04.644804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470f9f7-57df-41bb-a9d9-967464eab9cf admin admin] PATH_INFO: `/v3/projects/a808f54737d241aab1e766382c08e9dd/users/716d70da49784b7a9c682d8320851ce3/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:04.659152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470f9f7-57df-41bb-a9d9-967464eab9cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a808f54737d241aab1e766382c08e9dd, user_id=716d70da49784b7a9c682d8320851ce3, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:04.661883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470f9f7-57df-41bb-a9d9-967464eab9cf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:04.679325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 977/1937] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:04 2022] PUT /identity/v3/projects/a808f54737d241aab1e766382c08e9dd/users/716d70da49784b7a9c682d8320851ce3/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:04.689655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f2db7-2f12-4ccf-aa30-62bfd0aae9b4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:04.689940 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f2db7-2f12-4ccf-aa30-62bfd0aae9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:04.690247 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f2db7-2f12-4ccf-aa30-62bfd0aae9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:04.719150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e84f2db7-2f12-4ccf-aa30-62bfd0aae9b4 None None] MFA Rules not processed for user `716d70da49784b7a9c682d8320851ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:04.720422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e84f2db7-2f12-4ccf-aa30-62bfd0aae9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:04.729628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:04.788886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 961/1938] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:04.801234 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470f9f7-57df-41bb-a9d9-967464eab9cf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:04.804242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f470f9f7-57df-41bb-a9d9-967464eab9cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:04.867359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470f9f7-57df-41bb-a9d9-967464eab9cf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:04.871830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66353d3-752f-4ade-b978-75159ef4fd02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:04.872451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66353d3-752f-4ade-b978-75159ef4fd02 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:04.872727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66353d3-752f-4ade-b978-75159ef4fd02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:04.872995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66353d3-752f-4ade-b978-75159ef4fd02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:04.882380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66353d3-752f-4ade-b978-75159ef4fd02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:04.883813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66353d3-752f-4ade-b978-75159ef4fd02 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:04.887288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:04.894645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 978/1939] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:04.919227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f2db7-2f12-4ccf-aa30-62bfd0aae9b4 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:04.981127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f2db7-2f12-4ccf-aa30-62bfd0aae9b4 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:04.986518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:04.987143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:04.987416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:04.987717 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] PATH_INFO: `/v3/users/716d70da49784b7a9c682d8320851ce3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:04.988917 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=716d70da49784b7a9c682d8320851ce3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:04.990933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:05.040545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '716d70da49784b7a9c682d8320851ce3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:05.048790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '716d70da49784b7a9c682d8320851ce3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:05.049531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '716d70da49784b7a9c682d8320851ce3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:05.056904 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '716d70da49784b7a9c682d8320851ce3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:05.070340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '716d70da49784b7a9c682d8320851ce3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:05.081285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 962/1940] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:04 2022] DELETE /identity/v3/users/716d70da49784b7a9c682d8320851ce3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:05.082914 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6c978-ac2f-4dea-8438-9598f27e76eb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:05.083438 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6c978-ac2f-4dea-8438-9598f27e76eb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:05.083438 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6c978-ac2f-4dea-8438-9598f27e76eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:05.106856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c9e6c978-ac2f-4dea-8438-9598f27e76eb None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:05.108069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c9e6c978-ac2f-4dea-8438-9598f27e76eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:05.170033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 979/1941] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:05.182263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:05.183709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:05.223898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:05.251070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97b70b-93d8-40e0-86df-3d20cccda027 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:05.255064 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb8f6f-7282-43ea-9849-ed1040c051a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:05.255679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb8f6f-7282-43ea-9849-ed1040c051a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:05.255952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb8f6f-7282-43ea-9849-ed1040c051a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:05.256224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb8f6f-7282-43ea-9849-ed1040c051a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:05.265098 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb8f6f-7282-43ea-9849-ed1040c051a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:05.266575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb8f6f-7282-43ea-9849-ed1040c051a6 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:05.275796 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 963/1942] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:05.606004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6c978-ac2f-4dea-8438-9598f27e76eb None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:05.607575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c9e6c978-ac2f-4dea-8438-9598f27e76eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:05.623852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6c978-ac2f-4dea-8438-9598f27e76eb None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:05.631585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382383b6-2bad-45dc-9fa1-f023ff97d5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:05.632302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382383b6-2bad-45dc-9fa1-f023ff97d5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:05.632661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382383b6-2bad-45dc-9fa1-f023ff97d5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:05.633554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382383b6-2bad-45dc-9fa1-f023ff97d5f0 admin admin] PATH_INFO: `/v3/projects/a808f54737d241aab1e766382c08e9dd` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:05.634761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382383b6-2bad-45dc-9fa1-f023ff97d5f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a808f54737d241aab1e766382c08e9dd)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:05.636683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382383b6-2bad-45dc-9fa1-f023ff97d5f0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:05.704343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-382383b6-2bad-45dc-9fa1-f023ff97d5f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a808f54737d241aab1e766382c08e9dd', 'request_id': 'req-382383b6-2bad-45dc-9fa1-f023ff97d5f0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:05.717719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-382383b6-2bad-45dc-9fa1-f023ff97d5f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a808f54737d241aab1e766382c08e9dd', 'request_id': 'req-382383b6-2bad-45dc-9fa1-f023ff97d5f0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:05.724557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-382383b6-2bad-45dc-9fa1-f023ff97d5f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a808f54737d241aab1e766382c08e9dd', 'request_id': 'req-382383b6-2bad-45dc-9fa1-f023ff97d5f0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:05.738461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-382383b6-2bad-45dc-9fa1-f023ff97d5f0 admin admin] The token cache is being invalidate because project a808f54737d241aab1e766382c08e9dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:05.738754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-382383b6-2bad-45dc-9fa1-f023ff97d5f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:05.739412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-382383b6-2bad-45dc-9fa1-f023ff97d5f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:05.740972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 980/1943] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:05 2022] DELETE /identity/v3/projects/a808f54737d241aab1e766382c08e9dd => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:05.748743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190a926-a8ed-4722-94ef-604b32cfe5d7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:05.748743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190a926-a8ed-4722-94ef-604b32cfe5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:05.749346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190a926-a8ed-4722-94ef-604b32cfe5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:05.769779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7190a926-a8ed-4722-94ef-604b32cfe5d7 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:05.770939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7190a926-a8ed-4722-94ef-604b32cfe5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:05.832968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 964/1944] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:05.842032 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d5d2e-9d06-459a-a0f8-20814bcefec1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:05.842309 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d5d2e-9d06-459a-a0f8-20814bcefec1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:05.842585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d5d2e-9d06-459a-a0f8-20814bcefec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:05.862446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-055d5d2e-9d06-459a-a0f8-20814bcefec1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:05.863633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-055d5d2e-9d06-459a-a0f8-20814bcefec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:05.872017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 981/1945] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:05.876152 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190a926-a8ed-4722-94ef-604b32cfe5d7 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:05.886141 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190a926-a8ed-4722-94ef-604b32cfe5d7 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:05.892464 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f78a7-27f6-43a2-b955-e95ee7775cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:05.893119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f78a7-27f6-43a2-b955-e95ee7775cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:05.893431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f78a7-27f6-43a2-b955-e95ee7775cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:05.893697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f78a7-27f6-43a2-b955-e95ee7775cd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:05.894341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f78a7-27f6-43a2-b955-e95ee7775cd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:29:05.894641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f78a7-27f6-43a2-b955-e95ee7775cd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:05.896222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f78a7-27f6-43a2-b955-e95ee7775cd3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:05.908972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 965/1946] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:29:05.913518 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d5d2e-9d06-459a-a0f8-20814bcefec1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:05.923749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d5d2e-9d06-459a-a0f8-20814bcefec1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:05.929006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763a5b7-b04e-44f6-8a8f-9810264dc96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:05.929667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763a5b7-b04e-44f6-8a8f-9810264dc96f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:05.929944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763a5b7-b04e-44f6-8a8f-9810264dc96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:05.930212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763a5b7-b04e-44f6-8a8f-9810264dc96f admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:05.930898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763a5b7-b04e-44f6-8a8f-9810264dc96f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:05.932836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763a5b7-b04e-44f6-8a8f-9810264dc96f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:05.960744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 982/1947] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:05 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:05.965155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f78a7-27f6-43a2-b955-e95ee7775cd3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:05.975277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f78a7-27f6-43a2-b955-e95ee7775cd3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:06.031738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337de5f-4268-4cf0-bcb8-15f9c98010af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:06.032369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337de5f-4268-4cf0-bcb8-15f9c98010af admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:06.032622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337de5f-4268-4cf0-bcb8-15f9c98010af admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:06.032903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337de5f-4268-4cf0-bcb8-15f9c98010af admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:06.033558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337de5f-4268-4cf0-bcb8-15f9c98010af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:06.035471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337de5f-4268-4cf0-bcb8-15f9c98010af admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:06.059068 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 966/1948] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:05 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:29:06.063500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763a5b7-b04e-44f6-8a8f-9810264dc96f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:06.074997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763a5b7-b04e-44f6-8a8f-9810264dc96f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:06.080722 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f23a19-412f-4987-828d-58484521bb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:06.081381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f23a19-412f-4987-828d-58484521bb6e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:06.081653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f23a19-412f-4987-828d-58484521bb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:06.081922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f23a19-412f-4987-828d-58484521bb6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:06.082571 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f23a19-412f-4987-828d-58484521bb6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:06.084328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f23a19-412f-4987-828d-58484521bb6e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:06.095150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 983/1949] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:06.099700 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337de5f-4268-4cf0-bcb8-15f9c98010af admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:06.110767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337de5f-4268-4cf0-bcb8-15f9c98010af admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:06.116536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed644aae-b016-49f3-ae5f-38cc17fbf79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:06.117184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed644aae-b016-49f3-ae5f-38cc17fbf79e admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:06.117450 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed644aae-b016-49f3-ae5f-38cc17fbf79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:06.117720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed644aae-b016-49f3-ae5f-38cc17fbf79e admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:06.118381 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed644aae-b016-49f3-ae5f-38cc17fbf79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:06.119948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed644aae-b016-49f3-ae5f-38cc17fbf79e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:06.132558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 967/1950] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:06.136866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f23a19-412f-4987-828d-58484521bb6e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:06.147434 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f23a19-412f-4987-828d-58484521bb6e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:06.152702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b161b7-2edc-409e-acc2-01f705de925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:06.153422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b161b7-2edc-409e-acc2-01f705de925c admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:06.153767 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b161b7-2edc-409e-acc2-01f705de925c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:06.154135 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b161b7-2edc-409e-acc2-01f705de925c admin admin] PATH_INFO: `/v3/projects/f77952de9de34e068633312fa83f3c0c/users/56da4ed47ba4435d89841bb49f386bcf/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:06.169197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b161b7-2edc-409e-acc2-01f705de925c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77952de9de34e068633312fa83f3c0c, user_id=56da4ed47ba4435d89841bb49f386bcf, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:06.171878 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b161b7-2edc-409e-acc2-01f705de925c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:06.187060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 984/1951] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:06 2022] PUT /identity/v3/projects/f77952de9de34e068633312fa83f3c0c/users/56da4ed47ba4435d89841bb49f386bcf/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:06.197619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc330eaf-da62-4fd8-8aec-dcae5e3313ef None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:06.198001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc330eaf-da62-4fd8-8aec-dcae5e3313ef None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:06.198333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc330eaf-da62-4fd8-8aec-dcae5e3313ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:06.227180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cc330eaf-da62-4fd8-8aec-dcae5e3313ef None None] MFA Rules not processed for user `56da4ed47ba4435d89841bb49f386bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:06.228474 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cc330eaf-da62-4fd8-8aec-dcae5e3313ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:06.255309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:06.294732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 968/1952] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:06.310886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b161b7-2edc-409e-acc2-01f705de925c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:06.310886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d9b161b7-2edc-409e-acc2-01f705de925c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:06.328272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:06.367182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b161b7-2edc-409e-acc2-01f705de925c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:06.371162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f48632-b054-4bba-82b7-1531471ec284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:06.371753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f48632-b054-4bba-82b7-1531471ec284 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:06.372024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f48632-b054-4bba-82b7-1531471ec284 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:06.372299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f48632-b054-4bba-82b7-1531471ec284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:06.380834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f48632-b054-4bba-82b7-1531471ec284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:06.382234 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f48632-b054-4bba-82b7-1531471ec284 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:06.391285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 985/1953] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:14.396299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715473e-1f21-4a2e-986f-fa7504d76766 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:14.396810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715473e-1f21-4a2e-986f-fa7504d76766 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:14.396903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715473e-1f21-4a2e-986f-fa7504d76766 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:14.398751 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 969/1954] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:29:14.411357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced564d-4303-4f93-be3e-b439594fca72 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:14.411357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced564d-4303-4f93-be3e-b439594fca72 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:14.412159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced564d-4303-4f93-be3e-b439594fca72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:14.436469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9ced564d-4303-4f93-be3e-b439594fca72 None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:14.438380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9ced564d-4303-4f93-be3e-b439594fca72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:14.582448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 986/1955] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:29:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:14.596795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715473e-1f21-4a2e-986f-fa7504d76766 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:14.599116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7715473e-1f21-4a2e-986f-fa7504d76766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:14.635627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715473e-1f21-4a2e-986f-fa7504d76766 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:14.642722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24873a-aa96-44d8-bda4-307324f95c2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:14.643349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24873a-aa96-44d8-bda4-307324f95c2e service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:14.643623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24873a-aa96-44d8-bda4-307324f95c2e service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:14.643888 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24873a-aa96-44d8-bda4-307324f95c2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:14.653684 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24873a-aa96-44d8-bda4-307324f95c2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:14.655473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24873a-aa96-44d8-bda4-307324f95c2e service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:14.666698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 970/1956] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:24.570642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=49 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:24.667427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=21 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:29.814015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced564d-4303-4f93-be3e-b439594fca72 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:29.815630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9ced564d-4303-4f93-be3e-b439594fca72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:29.894681 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced564d-4303-4f93-be3e-b439594fca72 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:29.900378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:29.901003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:29.901317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:29.901588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] PATH_INFO: `/v3/users/83d2e57ea7084aa1ac94ed1571ffb91c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:29.902736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d2e57ea7084aa1ac94ed1571ffb91c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:29.904657 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:29.952769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d2e57ea7084aa1ac94ed1571ffb91c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:29.961379 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d2e57ea7084aa1ac94ed1571ffb91c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:29.961996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d2e57ea7084aa1ac94ed1571ffb91c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:29.969312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d2e57ea7084aa1ac94ed1571ffb91c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:29.983984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d2e57ea7084aa1ac94ed1571ffb91c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:29.992562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 987/1957] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:29 2022] DELETE /identity/v3/users/83d2e57ea7084aa1ac94ed1571ffb91c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:29.996815 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c22c41-afe6-4787-8430-bac5ce52bf0a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:29.997414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c22c41-afe6-4787-8430-bac5ce52bf0a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:29.997414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c22c41-afe6-4787-8430-bac5ce52bf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:30.018291 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-04c22c41-afe6-4787-8430-bac5ce52bf0a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:30.019613 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-04c22c41-afe6-4787-8430-bac5ce52bf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:30.122900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 971/1958] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:30.135620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:30.138358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:30.269778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b24e6-e810-435e-8ad2-c17a55a9c5f5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:30.275006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa64fc-79de-4fc0-a562-87358707df5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:30.275948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa64fc-79de-4fc0-a562-87358707df5c service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:30.276503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa64fc-79de-4fc0-a562-87358707df5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:30.276706 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa64fc-79de-4fc0-a562-87358707df5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:30.297006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa64fc-79de-4fc0-a562-87358707df5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:30.298536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa64fc-79de-4fc0-a562-87358707df5c service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:30.318357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 988/1959] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:31.130954 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c22c41-afe6-4787-8430-bac5ce52bf0a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:31.133125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-04c22c41-afe6-4787-8430-bac5ce52bf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:31.149050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c22c41-afe6-4787-8430-bac5ce52bf0a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:31.166440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:31.167367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:31.167861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:31.168071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] PATH_INFO: `/v3/projects/b4f996b865f24db596b2d8abf3a19634` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:31.169728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4f996b865f24db596b2d8abf3a19634)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:31.177523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:31.291435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4f996b865f24db596b2d8abf3a19634', 'request_id': 'req-b92b617c-0c64-42e1-a9bd-926fff49dfd0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:31.302387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4f996b865f24db596b2d8abf3a19634', 'request_id': 'req-b92b617c-0c64-42e1-a9bd-926fff49dfd0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:31.308550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4f996b865f24db596b2d8abf3a19634', 'request_id': 'req-b92b617c-0c64-42e1-a9bd-926fff49dfd0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:31.321971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] The token cache is being invalidate because project b4f996b865f24db596b2d8abf3a19634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:31.322484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:31.323362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:31.325258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 972/1960] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:31 2022] DELETE /identity/v3/projects/b4f996b865f24db596b2d8abf3a19634 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:31.329578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa64fc-79de-4fc0-a562-87358707df5c service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:31.339449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa64fc-79de-4fc0-a562-87358707df5c service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:31.359034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:31.459588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:31.460551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:31.460918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:31.461381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] PATH_INFO: `/v3/users/88c6e0efbe7d46a195fae8ea632118f9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:31.463454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88c6e0efbe7d46a195fae8ea632118f9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:31.466720 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:31.535685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88c6e0efbe7d46a195fae8ea632118f9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:31.546588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88c6e0efbe7d46a195fae8ea632118f9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:31.547406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88c6e0efbe7d46a195fae8ea632118f9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:31.555267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88c6e0efbe7d46a195fae8ea632118f9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:31.567735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88c6e0efbe7d46a195fae8ea632118f9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:31.574065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 989/1961] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:31 2022] DELETE /identity/v3/users/88c6e0efbe7d46a195fae8ea632118f9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:31.855182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:31.857051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:31.871349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b617c-0c64-42e1-a9bd-926fff49dfd0 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:31.933105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:31.933758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:31.934043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:31.934307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9 admin admin] PATH_INFO: `/v3/projects/7d02823d1691441e9c7778e45f643266` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:31.935459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d02823d1691441e9c7778e45f643266)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:31.937409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:31.968856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:31.991634 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d02823d1691441e9c7778e45f643266', 'request_id': 'req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:31.997536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:31.999224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:32.003754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d02823d1691441e9c7778e45f643266', 'request_id': 'req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:32.011803 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d02823d1691441e9c7778e45f643266', 'request_id': 'req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:32.015351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8fb61-fbe1-4fe0-aac3-be54d974b008 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:32.034147 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9 admin admin] The token cache is being invalidate because project 7d02823d1691441e9c7778e45f643266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:32.034928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:32.035805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d6e3f2b-d182-41ee-8485-474e6b7c9dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:32.037625 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 973/1962] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:31 2022] DELETE /identity/v3/projects/7d02823d1691441e9c7778e45f643266 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:32.043654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b9522-0888-4d7e-9ce1-7ff5c72addbd None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:32.044008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b9522-0888-4d7e-9ce1-7ff5c72addbd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:32.044369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b9522-0888-4d7e-9ce1-7ff5c72addbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:32.071897 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e03b9522-0888-4d7e-9ce1-7ff5c72addbd None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:32.073693 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e03b9522-0888-4d7e-9ce1-7ff5c72addbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:32.098967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e3ebd-5b77-4e2f-932a-dfb5c89e25df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:32.099536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e3ebd-5b77-4e2f-932a-dfb5c89e25df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:32.099976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e3ebd-5b77-4e2f-932a-dfb5c89e25df admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:32.100165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e3ebd-5b77-4e2f-932a-dfb5c89e25df admin admin] PATH_INFO: `/v3/users/56da4ed47ba4435d89841bb49f386bcf` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:32.101966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e3ebd-5b77-4e2f-932a-dfb5c89e25df admin admin] RBAC: Authorizing `identity:delete_user(user_id=56da4ed47ba4435d89841bb49f386bcf)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:32.104367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e3ebd-5b77-4e2f-932a-dfb5c89e25df admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:32.137521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 974/1963] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:32.146153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16e666-cd2c-4586-a25a-a1da39ebc0ae None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:32.146447 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16e666-cd2c-4586-a25a-a1da39ebc0ae None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:32.146721 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16e666-cd2c-4586-a25a-a1da39ebc0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:32.156429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:32.159189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7c0e3ebd-5b77-4e2f-932a-dfb5c89e25df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56da4ed47ba4435d89841bb49f386bcf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:32.166656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ee16e666-cd2c-4586-a25a-a1da39ebc0ae None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:32.167970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ee16e666-cd2c-4586-a25a-a1da39ebc0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:32.168161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7c0e3ebd-5b77-4e2f-932a-dfb5c89e25df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56da4ed47ba4435d89841bb49f386bcf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:32.168916 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7c0e3ebd-5b77-4e2f-932a-dfb5c89e25df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56da4ed47ba4435d89841bb49f386bcf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:32.176119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 975/1964] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:29:32.176372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7c0e3ebd-5b77-4e2f-932a-dfb5c89e25df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56da4ed47ba4435d89841bb49f386bcf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:32.180345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16e666-cd2c-4586-a25a-a1da39ebc0ae None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:32.188902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7c0e3ebd-5b77-4e2f-932a-dfb5c89e25df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56da4ed47ba4435d89841bb49f386bcf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:32.189823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16e666-cd2c-4586-a25a-a1da39ebc0ae None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:32.194584 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58d2b4-a10a-42ed-9b27-8b67da4339b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:32.195184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58d2b4-a10a-42ed-9b27-8b67da4339b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:32.195469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 990/1965] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:31 2022] DELETE /identity/v3/users/56da4ed47ba4435d89841bb49f386bcf => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:32.198557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58d2b4-a10a-42ed-9b27-8b67da4339b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:32.198870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58d2b4-a10a-42ed-9b27-8b67da4339b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:32.199565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58d2b4-a10a-42ed-9b27-8b67da4339b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:29:32.200059 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38c6e9-1d18-491f-9e58-69e043510953 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:32.200227 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58d2b4-a10a-42ed-9b27-8b67da4339b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:32.200334 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38c6e9-1d18-491f-9e58-69e043510953 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:32.200577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38c6e9-1d18-491f-9e58-69e043510953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:32.201761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58d2b4-a10a-42ed-9b27-8b67da4339b5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:32.220212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 976/1966] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:29:32.221334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58d2b4-a10a-42ed-9b27-8b67da4339b5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:32.222399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2b38c6e9-1d18-491f-9e58-69e043510953 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:32.224100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2b38c6e9-1d18-491f-9e58-69e043510953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:32.231373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58d2b4-a10a-42ed-9b27-8b67da4339b5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:32.289369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 991/1967] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:32.295119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4214fe-9126-4dc8-9a70-e71b0aca2eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:32.295747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4214fe-9126-4dc8-9a70-e71b0aca2eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:32.296026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4214fe-9126-4dc8-9a70-e71b0aca2eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:32.296296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4214fe-9126-4dc8-9a70-e71b0aca2eb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:32.296959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4214fe-9126-4dc8-9a70-e71b0aca2eb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:32.299293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4214fe-9126-4dc8-9a70-e71b0aca2eb3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:32.304364 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38c6e9-1d18-491f-9e58-69e043510953 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:32.305737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2b38c6e9-1d18-491f-9e58-69e043510953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:32.337244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 977/1968] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:32 2022] POST /identity/v3/projects => generated 357 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:32.342008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4214fe-9126-4dc8-9a70-e71b0aca2eb3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:32.368195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:32.375297 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38c6e9-1d18-491f-9e58-69e043510953 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:32.436105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4214fe-9126-4dc8-9a70-e71b0aca2eb3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:32.442593 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e043b-0db0-42a8-9345-cebdef90a32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:32.443315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e043b-0db0-42a8-9345-cebdef90a32e admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:32.443735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e043b-0db0-42a8-9345-cebdef90a32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:32.444033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e043b-0db0-42a8-9345-cebdef90a32e admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:32.444988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e043b-0db0-42a8-9345-cebdef90a32e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:32.446679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e043b-0db0-42a8-9345-cebdef90a32e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:32.462835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d83ad4-5c98-4d8a-b54f-3487b364fa19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:32.463591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d83ad4-5c98-4d8a-b54f-3487b364fa19 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:32.463772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d83ad4-5c98-4d8a-b54f-3487b364fa19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:32.463986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d83ad4-5c98-4d8a-b54f-3487b364fa19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:32.480799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 978/1969] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:32 2022] POST /identity/v3/users => generated 279 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:29:32.485444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e043b-0db0-42a8-9345-cebdef90a32e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:32.495708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e043b-0db0-42a8-9345-cebdef90a32e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:32.499285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d83ad4-5c98-4d8a-b54f-3487b364fa19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:32.500724 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f1b68-2c2f-49a2-a483-d48c97b92f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:32.500889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d83ad4-5c98-4d8a-b54f-3487b364fa19 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:32.501401 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f1b68-2c2f-49a2-a483-d48c97b92f24 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:32.501738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f1b68-2c2f-49a2-a483-d48c97b92f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:32.502135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f1b68-2c2f-49a2-a483-d48c97b92f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:32.502780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f1b68-2c2f-49a2-a483-d48c97b92f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:32.504348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f1b68-2c2f-49a2-a483-d48c97b92f24 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:32.522094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 992/1970] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:32.524278 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 979/1971] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:29:32.524753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d83ad4-5c98-4d8a-b54f-3487b364fa19 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:32.539755 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d83ad4-5c98-4d8a-b54f-3487b364fa19 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:32.549282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed111c-6a6c-4734-af1a-485491624156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:32.551677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed111c-6a6c-4734-af1a-485491624156 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:32.553389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed111c-6a6c-4734-af1a-485491624156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:32.553713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed111c-6a6c-4734-af1a-485491624156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:32.563352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed111c-6a6c-4734-af1a-485491624156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:32.563352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed111c-6a6c-4734-af1a-485491624156 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:32.570148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 993/1972] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:29:32.575098 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f1b68-2c2f-49a2-a483-d48c97b92f24 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:32.590567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f1b68-2c2f-49a2-a483-d48c97b92f24 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:32.596588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39821c9-762b-4114-9491-dda0f1f9e9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:32.597446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39821c9-762b-4114-9491-dda0f1f9e9cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:32.597889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39821c9-762b-4114-9491-dda0f1f9e9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:32.599802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39821c9-762b-4114-9491-dda0f1f9e9cb admin admin] PATH_INFO: `/v3/projects/c06f3fd72127413294ab1c0af157c399/users/b8e95b8d56bc46eca9f9f828ddf9ab17/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:32.625668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39821c9-762b-4114-9491-dda0f1f9e9cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c06f3fd72127413294ab1c0af157c399, user_id=b8e95b8d56bc46eca9f9f828ddf9ab17, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:32.630003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39821c9-762b-4114-9491-dda0f1f9e9cb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:32.670139 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 980/1973] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:32 2022] PUT /identity/v3/projects/c06f3fd72127413294ab1c0af157c399/users/b8e95b8d56bc46eca9f9f828ddf9ab17/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:32.671788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6a786-ecb4-468b-8a61-b3c03cde321c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:32.672547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6a786-ecb4-468b-8a61-b3c03cde321c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:32.672905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6a786-ecb4-468b-8a61-b3c03cde321c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:32.706174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:32.718560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b0a6a786-ecb4-468b-8a61-b3c03cde321c None None] MFA Rules not processed for user `b8e95b8d56bc46eca9f9f828ddf9ab17`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:32.719856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b0a6a786-ecb4-468b-8a61-b3c03cde321c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:32.808994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 994/1974] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:32.947002 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39821c9-762b-4114-9491-dda0f1f9e9cb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:32.948990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d39821c9-762b-4114-9491-dda0f1f9e9cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:33.031084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39821c9-762b-4114-9491-dda0f1f9e9cb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:33.037648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a6614-5986-468a-982b-b0e72f58a500 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:33.038310 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a6614-5986-468a-982b-b0e72f58a500 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.038670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a6614-5986-468a-982b-b0e72f58a500 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.039069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a6614-5986-468a-982b-b0e72f58a500 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.049509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a6614-5986-468a-982b-b0e72f58a500 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:33.051049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a6614-5986-468a-982b-b0e72f58a500 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:33.064239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 981/1975] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:33.082515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6a786-ecb4-468b-8a61-b3c03cde321c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:33.091268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a6614-5986-468a-982b-b0e72f58a500 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:33.095867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-453a6614-5986-468a-982b-b0e72f58a500 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:33.167818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6a786-ecb4-468b-8a61-b3c03cde321c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:33.173336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a6614-5986-468a-982b-b0e72f58a500 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:33.174378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00c517-30ce-4692-89ec-d16984e4c6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:33.175109 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00c517-30ce-4692-89ec-d16984e4c6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.175540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00c517-30ce-4692-89ec-d16984e4c6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.175878 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00c517-30ce-4692-89ec-d16984e4c6a4 admin admin] PATH_INFO: `/v3/users/b8e95b8d56bc46eca9f9f828ddf9ab17` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.177119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00c517-30ce-4692-89ec-d16984e4c6a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8e95b8d56bc46eca9f9f828ddf9ab17)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:33.178986 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:33.179103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00c517-30ce-4692-89ec-d16984e4c6a4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:33.179676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.179963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.180230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2 admin admin] PATH_INFO: `/v3/projects/f77952de9de34e068633312fa83f3c0c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.181628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f77952de9de34e068633312fa83f3c0c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:33.183760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:33.212762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:33.237936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7f00c517-30ce-4692-89ec-d16984e4c6a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8e95b8d56bc46eca9f9f828ddf9ab17'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:33.247623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f77952de9de34e068633312fa83f3c0c', 'request_id': 'req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:33.251200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7f00c517-30ce-4692-89ec-d16984e4c6a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8e95b8d56bc46eca9f9f828ddf9ab17'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:33.251200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7f00c517-30ce-4692-89ec-d16984e4c6a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8e95b8d56bc46eca9f9f828ddf9ab17'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:33.259763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f77952de9de34e068633312fa83f3c0c', 'request_id': 'req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:33.260054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7f00c517-30ce-4692-89ec-d16984e4c6a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8e95b8d56bc46eca9f9f828ddf9ab17'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:33.267329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f77952de9de34e068633312fa83f3c0c', 'request_id': 'req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:33.275009 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7f00c517-30ce-4692-89ec-d16984e4c6a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8e95b8d56bc46eca9f9f828ddf9ab17'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:33.281309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2 admin admin] The token cache is being invalidate because project f77952de9de34e068633312fa83f3c0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:33.281860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:33.282532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7aa2a5b5-f2f2-48bf-aa1f-8a578a5cc7c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:33.283198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 995/1976] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:33 2022] DELETE /identity/v3/users/b8e95b8d56bc46eca9f9f828ddf9ab17 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:33.286136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 982/1977] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:33 2022] DELETE /identity/v3/projects/f77952de9de34e068633312fa83f3c0c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:33.287671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b9f8-e788-4d75-8b86-e57d1002f8e4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.288090 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b9f8-e788-4d75-8b86-e57d1002f8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.288506 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b9f8-e788-4d75-8b86-e57d1002f8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.290525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f63e2-4261-418d-af08-f2dcb070389e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.290970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f63e2-4261-418d-af08-f2dcb070389e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.291435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f63e2-4261-418d-af08-f2dcb070389e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.315223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e2b0b9f8-e788-4d75-8b86-e57d1002f8e4 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:33.317275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e2b0b9f8-e788-4d75-8b86-e57d1002f8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:33.320348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fa4f63e2-4261-418d-af08-f2dcb070389e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:33.322024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fa4f63e2-4261-418d-af08-f2dcb070389e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:33.333232 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:33.423941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 996/1978] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:33.433986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b9f8-e788-4d75-8b86-e57d1002f8e4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:33.435513 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e2b0b9f8-e788-4d75-8b86-e57d1002f8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:33.457426 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 983/1979] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:33.466222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0d44c-e8d4-4dcc-95e6-3462b3044692 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.466737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0d44c-e8d4-4dcc-95e6-3462b3044692 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.467242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0d44c-e8d4-4dcc-95e6-3462b3044692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.491488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cbd0d44c-e8d4-4dcc-95e6-3462b3044692 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:33.493235 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cbd0d44c-e8d4-4dcc-95e6-3462b3044692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:33.502885 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 984/1980] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:29:33.507727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0d44c-e8d4-4dcc-95e6-3462b3044692 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:33.514871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b9f8-e788-4d75-8b86-e57d1002f8e4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:33.517646 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0d44c-e8d4-4dcc-95e6-3462b3044692 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:33.521030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27ceaa-83a1-4b33-a4e0-5d4b15c55153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:33.521683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27ceaa-83a1-4b33-a4e0-5d4b15c55153 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.521994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27ceaa-83a1-4b33-a4e0-5d4b15c55153 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.522500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27ceaa-83a1-4b33-a4e0-5d4b15c55153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.524019 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f814c-682d-4689-a1ee-be31dde7aba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:33.525006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f814c-682d-4689-a1ee-be31dde7aba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.525430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f814c-682d-4689-a1ee-be31dde7aba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.525828 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f814c-682d-4689-a1ee-be31dde7aba8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.527251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f814c-682d-4689-a1ee-be31dde7aba8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:29:33.527251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f814c-682d-4689-a1ee-be31dde7aba8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:33.528882 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f814c-682d-4689-a1ee-be31dde7aba8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:33.535272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27ceaa-83a1-4b33-a4e0-5d4b15c55153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:33.537431 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27ceaa-83a1-4b33-a4e0-5d4b15c55153 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:33.544478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 985/1981] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:29:33.545542 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f814c-682d-4689-a1ee-be31dde7aba8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:33.548304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 997/1982] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:29:33.555710 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f814c-682d-4689-a1ee-be31dde7aba8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:33.561731 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0f280-d453-4d76-9445-84f0be1cbf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:33.574101 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0f280-d453-4d76-9445-84f0be1cbf9c admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.574437 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0f280-d453-4d76-9445-84f0be1cbf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.577376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0f280-d453-4d76-9445-84f0be1cbf9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.579086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0f280-d453-4d76-9445-84f0be1cbf9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:33.582119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0f280-d453-4d76-9445-84f0be1cbf9c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:33.632669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 986/1983] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:33 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:29:33.636865 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27ceaa-83a1-4b33-a4e0-5d4b15c55153 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:33.648072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27ceaa-83a1-4b33-a4e0-5d4b15c55153 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:33.715522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6cdd8-7f1e-4fed-a151-db5680b86886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:33.715522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6cdd8-7f1e-4fed-a151-db5680b86886 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.715522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6cdd8-7f1e-4fed-a151-db5680b86886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.725619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6cdd8-7f1e-4fed-a151-db5680b86886 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.726896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6cdd8-7f1e-4fed-a151-db5680b86886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:33.729928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6cdd8-7f1e-4fed-a151-db5680b86886 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:33.753152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 998/1984] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:33 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:33.757414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0f280-d453-4d76-9445-84f0be1cbf9c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:33.768576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0f280-d453-4d76-9445-84f0be1cbf9c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:33.773449 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e8039-821e-4b15-abcd-f1db30d7127e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:33.774137 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e8039-821e-4b15-abcd-f1db30d7127e admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.774486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e8039-821e-4b15-abcd-f1db30d7127e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.774823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e8039-821e-4b15-abcd-f1db30d7127e admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.775547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e8039-821e-4b15-abcd-f1db30d7127e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:33.777177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e8039-821e-4b15-abcd-f1db30d7127e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:33.787509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 987/1985] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:29:33.791707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6cdd8-7f1e-4fed-a151-db5680b86886 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:33.801837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6cdd8-7f1e-4fed-a151-db5680b86886 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:33.806456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516cb32-7f81-44c6-bb5b-fd3784152e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:33.807066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516cb32-7f81-44c6-bb5b-fd3784152e01 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.807469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516cb32-7f81-44c6-bb5b-fd3784152e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.807747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516cb32-7f81-44c6-bb5b-fd3784152e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.808434 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516cb32-7f81-44c6-bb5b-fd3784152e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:33.810065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516cb32-7f81-44c6-bb5b-fd3784152e01 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:33.820188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 999/1986] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:33.824327 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e8039-821e-4b15-abcd-f1db30d7127e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:33.835633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e8039-821e-4b15-abcd-f1db30d7127e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:33.841340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9e80c-839a-4f0f-a49c-0f5d5b56f884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:33.842075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9e80c-839a-4f0f-a49c-0f5d5b56f884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.842377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9e80c-839a-4f0f-a49c-0f5d5b56f884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.842606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9e80c-839a-4f0f-a49c-0f5d5b56f884 admin admin] PATH_INFO: `/v3/projects/4ca16cc178624419954df6bad327f722/users/f6a5aff543234d7a9976c2e43453aef2/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.859881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9e80c-839a-4f0f-a49c-0f5d5b56f884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca16cc178624419954df6bad327f722, user_id=f6a5aff543234d7a9976c2e43453aef2, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:33.862779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9e80c-839a-4f0f-a49c-0f5d5b56f884 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:33.877462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 988/1987] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:33 2022] PUT /identity/v3/projects/4ca16cc178624419954df6bad327f722/users/f6a5aff543234d7a9976c2e43453aef2/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:33.882956 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d81df-eae1-4b72-8a35-fc5ef5fd4de6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:33.883330 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d81df-eae1-4b72-8a35-fc5ef5fd4de6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:33.883639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d81df-eae1-4b72-8a35-fc5ef5fd4de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:33.905907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-259d81df-eae1-4b72-8a35-fc5ef5fd4de6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:33.907162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-259d81df-eae1-4b72-8a35-fc5ef5fd4de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:33.919477 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9e80c-839a-4f0f-a49c-0f5d5b56f884 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:33.921369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6ee9e80c-839a-4f0f-a49c-0f5d5b56f884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:33.977491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:33.979994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:34.020060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9e80c-839a-4f0f-a49c-0f5d5b56f884 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:34.024984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa3a94-ccf3-43a3-a26b-9a88638128c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:34.025661 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa3a94-ccf3-43a3-a26b-9a88638128c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:34.027696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa3a94-ccf3-43a3-a26b-9a88638128c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:34.027696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa3a94-ccf3-43a3-a26b-9a88638128c7 admin admin] PATH_INFO: `/v3/projects/c06f3fd72127413294ab1c0af157c399` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:34.028071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa3a94-ccf3-43a3-a26b-9a88638128c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c06f3fd72127413294ab1c0af157c399)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:34.032727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1000/1988] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:34.033489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa3a94-ccf3-43a3-a26b-9a88638128c7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:34.044378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d81df-eae1-4b72-8a35-fc5ef5fd4de6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:34.106018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3baa3a94-ccf3-43a3-a26b-9a88638128c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c06f3fd72127413294ab1c0af157c399', 'request_id': 'req-3baa3a94-ccf3-43a3-a26b-9a88638128c7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:34.125011 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3baa3a94-ccf3-43a3-a26b-9a88638128c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c06f3fd72127413294ab1c0af157c399', 'request_id': 'req-3baa3a94-ccf3-43a3-a26b-9a88638128c7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:34.133848 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3baa3a94-ccf3-43a3-a26b-9a88638128c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c06f3fd72127413294ab1c0af157c399', 'request_id': 'req-3baa3a94-ccf3-43a3-a26b-9a88638128c7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:34.160344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d81df-eae1-4b72-8a35-fc5ef5fd4de6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:34.167562 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3baa3a94-ccf3-43a3-a26b-9a88638128c7 admin admin] The token cache is being invalidate because project c06f3fd72127413294ab1c0af157c399 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:34.167852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3baa3a94-ccf3-43a3-a26b-9a88638128c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:34.168535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3baa3a94-ccf3-43a3-a26b-9a88638128c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:34.173759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 989/1989] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:33 2022] DELETE /identity/v3/projects/c06f3fd72127413294ab1c0af157c399 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:34.175435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e077f4-c255-44bf-b09a-a4f564afc024 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:34.175718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e077f4-c255-44bf-b09a-a4f564afc024 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:34.175989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e077f4-c255-44bf-b09a-a4f564afc024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:34.239176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-74e077f4-c255-44bf-b09a-a4f564afc024 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:34.241932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-74e077f4-c255-44bf-b09a-a4f564afc024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:34.250679 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3993f-09b0-4e53-86ff-99d0d6ea4879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:34.251366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3993f-09b0-4e53-86ff-99d0d6ea4879 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:34.251748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3993f-09b0-4e53-86ff-99d0d6ea4879 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:34.253146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3993f-09b0-4e53-86ff-99d0d6ea4879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:34.255670 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5fa3993f-09b0-4e53-86ff-99d0d6ea4879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:34.359969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 990/1990] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:34.361850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1ac82-ad7d-4932-bf36-cbbe3454d71e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:34.362153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1ac82-ad7d-4932-bf36-cbbe3454d71e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:34.362591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1ac82-ad7d-4932-bf36-cbbe3454d71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:34.371725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3993f-09b0-4e53-86ff-99d0d6ea4879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:34.374122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3993f-09b0-4e53-86ff-99d0d6ea4879 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:34.387171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-dba1ac82-ad7d-4932-bf36-cbbe3454d71e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:34.390300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-dba1ac82-ad7d-4932-bf36-cbbe3454d71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:34.390927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1001/1991] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:34.405310 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 991/1992] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:34.406868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3993f-09b0-4e53-86ff-99d0d6ea4879 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:34.425592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3993f-09b0-4e53-86ff-99d0d6ea4879 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:34.431048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc749b-07a3-4f3b-8678-3e4d0f93bc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:34.432289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc749b-07a3-4f3b-8678-3e4d0f93bc4b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:34.432289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc749b-07a3-4f3b-8678-3e4d0f93bc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:34.432488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc749b-07a3-4f3b-8678-3e4d0f93bc4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:34.433468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc749b-07a3-4f3b-8678-3e4d0f93bc4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:29:34.433852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc749b-07a3-4f3b-8678-3e4d0f93bc4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:34.437485 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc749b-07a3-4f3b-8678-3e4d0f93bc4b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:34.469896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1002/1993] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:29:34.474719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1ac82-ad7d-4932-bf36-cbbe3454d71e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:34.494513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1ac82-ad7d-4932-bf36-cbbe3454d71e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:34.502074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf26a0-e030-468f-a1f7-6f5cbfbf2ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:34.502964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf26a0-e030-468f-a1f7-6f5cbfbf2ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:34.503304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf26a0-e030-468f-a1f7-6f5cbfbf2ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:34.503638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf26a0-e030-468f-a1f7-6f5cbfbf2ef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:34.504423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf26a0-e030-468f-a1f7-6f5cbfbf2ef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:34.506810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf26a0-e030-468f-a1f7-6f5cbfbf2ef8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:34.569226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 992/1994] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:34 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:34.575212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc749b-07a3-4f3b-8678-3e4d0f93bc4b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:34.595265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc749b-07a3-4f3b-8678-3e4d0f93bc4b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:34.641237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:34.690049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3a98b-2b49-4794-8b5b-2568e7f03f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:34.690818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3a98b-2b49-4794-8b5b-2568e7f03f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:34.691195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3a98b-2b49-4794-8b5b-2568e7f03f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:34.691586 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3a98b-2b49-4794-8b5b-2568e7f03f0f admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:34.692354 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3a98b-2b49-4794-8b5b-2568e7f03f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:34.694448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3a98b-2b49-4794-8b5b-2568e7f03f0f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:34.729136 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1003/1995] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:34 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:34.731838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf26a0-e030-468f-a1f7-6f5cbfbf2ef8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:34.745818 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf26a0-e030-468f-a1f7-6f5cbfbf2ef8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:34.752795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb9e48-06cd-4c28-b52a-49c792582ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:34.753484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb9e48-06cd-4c28-b52a-49c792582ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:34.753761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb9e48-06cd-4c28-b52a-49c792582ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:34.754024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb9e48-06cd-4c28-b52a-49c792582ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:34.754671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb9e48-06cd-4c28-b52a-49c792582ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:34.756427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb9e48-06cd-4c28-b52a-49c792582ce8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:34.767762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 993/1996] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:34.771794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3a98b-2b49-4794-8b5b-2568e7f03f0f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:34.783642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3a98b-2b49-4794-8b5b-2568e7f03f0f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:34.788763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8cfc9-292a-4db7-8c7f-daf74a4659e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:34.789944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8cfc9-292a-4db7-8c7f-daf74a4659e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:34.790338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8cfc9-292a-4db7-8c7f-daf74a4659e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:34.790706 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8cfc9-292a-4db7-8c7f-daf74a4659e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:34.791636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8cfc9-292a-4db7-8c7f-daf74a4659e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:34.793455 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8cfc9-292a-4db7-8c7f-daf74a4659e8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:34.809580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1004/1997] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:29:34.811368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb9e48-06cd-4c28-b52a-49c792582ce8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:34.827697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb9e48-06cd-4c28-b52a-49c792582ce8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:34.833733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8733c-50fd-45d9-8edf-ac9f8e3461ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:34.834574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8733c-50fd-45d9-8edf-ac9f8e3461ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:34.835200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8733c-50fd-45d9-8edf-ac9f8e3461ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:34.835624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8733c-50fd-45d9-8edf-ac9f8e3461ad admin admin] PATH_INFO: `/v3/projects/a70ba5dbb7724b5cb0e99caf49e76f72/users/2d587668c17b4eba8b10bfa172548bca/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:34.858370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8733c-50fd-45d9-8edf-ac9f8e3461ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=a70ba5dbb7724b5cb0e99caf49e76f72, user_id=2d587668c17b4eba8b10bfa172548bca, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:34.861998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8733c-50fd-45d9-8edf-ac9f8e3461ad admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:34.873177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:34.890976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 994/1998] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:34 2022] PUT /identity/v3/projects/a70ba5dbb7724b5cb0e99caf49e76f72/users/2d587668c17b4eba8b10bfa172548bca/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:29:34.893332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274076f-a741-43cd-bfff-bf8214927ac2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:34.893788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274076f-a741-43cd-bfff-bf8214927ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:34.895075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274076f-a741-43cd-bfff-bf8214927ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:34.940586 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1274076f-a741-43cd-bfff-bf8214927ac2 None None] MFA Rules not processed for user `2d587668c17b4eba8b10bfa172548bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:34.942083 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1274076f-a741-43cd-bfff-bf8214927ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:35.028943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1005/1999] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:35.042044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8733c-50fd-45d9-8edf-ac9f8e3461ad admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:35.044679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-99c8733c-50fd-45d9-8edf-ac9f8e3461ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:35.112370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8733c-50fd-45d9-8edf-ac9f8e3461ad admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:35.118696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0cfcd-7e13-470e-9afb-6387549124d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:35.119497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0cfcd-7e13-470e-9afb-6387549124d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:35.119595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0cfcd-7e13-470e-9afb-6387549124d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:35.119850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0cfcd-7e13-470e-9afb-6387549124d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:35.130014 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0cfcd-7e13-470e-9afb-6387549124d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:35.131504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0cfcd-7e13-470e-9afb-6387549124d3 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:35.144010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 995/2000] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:35.874718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274076f-a741-43cd-bfff-bf8214927ac2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:35.877414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1274076f-a741-43cd-bfff-bf8214927ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:35.996890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274076f-a741-43cd-bfff-bf8214927ac2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:36.012650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:36.013905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:36.014780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:36.017564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] PATH_INFO: `/v3/users/800825703b0240a08e17e42e278ec607` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:36.020618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=800825703b0240a08e17e42e278ec607)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:36.026531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:36.155071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:36.165544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '800825703b0240a08e17e42e278ec607'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:36.185520 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '800825703b0240a08e17e42e278ec607'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:36.186219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '800825703b0240a08e17e42e278ec607'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:36.199967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '800825703b0240a08e17e42e278ec607'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:36.239567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '800825703b0240a08e17e42e278ec607'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:36.252213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1006/2001] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:35 2022] DELETE /identity/v3/users/800825703b0240a08e17e42e278ec607 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:36.257581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291281a1-73a8-466c-9260-b378b31751b9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:36.258071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291281a1-73a8-466c-9260-b378b31751b9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:36.258245 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291281a1-73a8-466c-9260-b378b31751b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:36.288176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-291281a1-73a8-466c-9260-b378b31751b9 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:36.291037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-291281a1-73a8-466c-9260-b378b31751b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:36.384334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 996/2002] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:36.391028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:36.392699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:36.509038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca484621-7d60-40f9-823b-cd4fce57bcf6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:36.513736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c12784-44db-43a8-b2d7-caf03653af05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:36.514365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c12784-44db-43a8-b2d7-caf03653af05 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:36.514640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c12784-44db-43a8-b2d7-caf03653af05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:36.522320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c12784-44db-43a8-b2d7-caf03653af05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:36.547953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c12784-44db-43a8-b2d7-caf03653af05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:36.550469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c12784-44db-43a8-b2d7-caf03653af05 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:36.563812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1007/2003] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:37.076409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291281a1-73a8-466c-9260-b378b31751b9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:37.078268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-291281a1-73a8-466c-9260-b378b31751b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:37.103443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291281a1-73a8-466c-9260-b378b31751b9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:37.117133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:37.117807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:37.118079 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:37.118368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] PATH_INFO: `/v3/projects/646448a4902f4565a093694056ddbd2d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:37.121875 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646448a4902f4565a093694056ddbd2d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:37.123805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:37.288807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646448a4902f4565a093694056ddbd2d', 'request_id': 'req-af72215e-548f-401a-83cb-68b52c3767d2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:37.310638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646448a4902f4565a093694056ddbd2d', 'request_id': 'req-af72215e-548f-401a-83cb-68b52c3767d2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:37.320346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646448a4902f4565a093694056ddbd2d', 'request_id': 'req-af72215e-548f-401a-83cb-68b52c3767d2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:37.346173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] The token cache is being invalidate because project 646448a4902f4565a093694056ddbd2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:37.346769 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:37.350770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:37.354168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 997/2004] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:37 2022] DELETE /identity/v3/projects/646448a4902f4565a093694056ddbd2d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:37.358903 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c12784-44db-43a8-b2d7-caf03653af05 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:37.374689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c12784-44db-43a8-b2d7-caf03653af05 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:37.523532 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca317ba-37de-4a20-9132-83de74c44ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:37.529410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca317ba-37de-4a20-9132-83de74c44ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:37.529410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca317ba-37de-4a20-9132-83de74c44ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:37.529410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca317ba-37de-4a20-9132-83de74c44ea0 admin admin] PATH_INFO: `/v3/users/14c5514eed194029aebe78922ece2e63` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:37.541690 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca317ba-37de-4a20-9132-83de74c44ea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14c5514eed194029aebe78922ece2e63)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:37.552875 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca317ba-37de-4a20-9132-83de74c44ea0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:37.696421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9ca317ba-37de-4a20-9132-83de74c44ea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14c5514eed194029aebe78922ece2e63'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:37.720424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9ca317ba-37de-4a20-9132-83de74c44ea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14c5514eed194029aebe78922ece2e63'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:37.721271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9ca317ba-37de-4a20-9132-83de74c44ea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14c5514eed194029aebe78922ece2e63'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:37.736814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9ca317ba-37de-4a20-9132-83de74c44ea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14c5514eed194029aebe78922ece2e63'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:37.741507 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:37.768398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9ca317ba-37de-4a20-9132-83de74c44ea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14c5514eed194029aebe78922ece2e63'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:37.780737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1008/2005] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:37 2022] DELETE /identity/v3/users/14c5514eed194029aebe78922ece2e63 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:38.338537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:38.341153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:38.383835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72215e-548f-401a-83cb-68b52c3767d2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:38.424133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:38.520215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47113f-ac12-4c74-a8b9-525249a4d7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:38.521061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47113f-ac12-4c74-a8b9-525249a4d7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:38.521434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47113f-ac12-4c74-a8b9-525249a4d7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:38.521768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47113f-ac12-4c74-a8b9-525249a4d7bf admin admin] PATH_INFO: `/v3/projects/b1a0a97dc69a468b95302dcd036766c6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:38.523141 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47113f-ac12-4c74-a8b9-525249a4d7bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1a0a97dc69a468b95302dcd036766c6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:38.525326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47113f-ac12-4c74-a8b9-525249a4d7bf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:38.622771 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5c47113f-ac12-4c74-a8b9-525249a4d7bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1a0a97dc69a468b95302dcd036766c6', 'request_id': 'req-5c47113f-ac12-4c74-a8b9-525249a4d7bf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:38.639763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5c47113f-ac12-4c74-a8b9-525249a4d7bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1a0a97dc69a468b95302dcd036766c6', 'request_id': 'req-5c47113f-ac12-4c74-a8b9-525249a4d7bf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:38.656634 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5c47113f-ac12-4c74-a8b9-525249a4d7bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1a0a97dc69a468b95302dcd036766c6', 'request_id': 'req-5c47113f-ac12-4c74-a8b9-525249a4d7bf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:38.681578 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5c47113f-ac12-4c74-a8b9-525249a4d7bf admin admin] The token cache is being invalidate because project b1a0a97dc69a468b95302dcd036766c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:38.681880 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5c47113f-ac12-4c74-a8b9-525249a4d7bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:38.682566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5c47113f-ac12-4c74-a8b9-525249a4d7bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:38.687577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 998/2006] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:38 2022] DELETE /identity/v3/projects/b1a0a97dc69a468b95302dcd036766c6 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:38.689824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a5324-22e1-4858-9173-c037595afbe4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:38.689824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a5324-22e1-4858-9173-c037595afbe4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:38.690390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a5324-22e1-4858-9173-c037595afbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:38.713516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-558a5324-22e1-4858-9173-c037595afbe4 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:38.714899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-558a5324-22e1-4858-9173-c037595afbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:38.791568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1009/2007] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:38.793711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a75e7b-f609-480a-b2ad-aa8f60a096e5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:38.794027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a75e7b-f609-480a-b2ad-aa8f60a096e5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:38.794304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a75e7b-f609-480a-b2ad-aa8f60a096e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:38.817339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-75a75e7b-f609-480a-b2ad-aa8f60a096e5 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:38.818833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-75a75e7b-f609-480a-b2ad-aa8f60a096e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:38.827214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 999/2008] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:38.832034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a5324-22e1-4858-9173-c037595afbe4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:38.851461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a5324-22e1-4858-9173-c037595afbe4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:38.857477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeea28a-652f-4055-bbb6-b8587c553236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:38.858418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeea28a-652f-4055-bbb6-b8587c553236 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:38.858845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeea28a-652f-4055-bbb6-b8587c553236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:38.859268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeea28a-652f-4055-bbb6-b8587c553236 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:38.860373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeea28a-652f-4055-bbb6-b8587c553236 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:29:38.860828 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeea28a-652f-4055-bbb6-b8587c553236 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:38.865606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeea28a-652f-4055-bbb6-b8587c553236 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:38.889619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1010/2009] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:29:38.894285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a75e7b-f609-480a-b2ad-aa8f60a096e5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:38.909004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a75e7b-f609-480a-b2ad-aa8f60a096e5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:38.914248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd3f4a-91e0-421e-aa13-68f8b4ea2b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:38.914878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd3f4a-91e0-421e-aa13-68f8b4ea2b0c admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:38.915149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd3f4a-91e0-421e-aa13-68f8b4ea2b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:38.915587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd3f4a-91e0-421e-aa13-68f8b4ea2b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:38.916504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd3f4a-91e0-421e-aa13-68f8b4ea2b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:38.919719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd3f4a-91e0-421e-aa13-68f8b4ea2b0c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:38.974271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeea28a-652f-4055-bbb6-b8587c553236 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:38.977747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bbeea28a-652f-4055-bbb6-b8587c553236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:38.983372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1000/2010] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:38 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:38.988338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd3f4a-91e0-421e-aa13-68f8b4ea2b0c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:39.004178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeea28a-652f-4055-bbb6-b8587c553236 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:39.080835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd3f4a-91e0-421e-aa13-68f8b4ea2b0c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:39.088437 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf39476-9640-4c49-bf3c-6995f4638133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:39.089556 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf39476-9640-4c49-bf3c-6995f4638133 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:39.090087 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf39476-9640-4c49-bf3c-6995f4638133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:39.090518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf39476-9640-4c49-bf3c-6995f4638133 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:39.091457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf39476-9640-4c49-bf3c-6995f4638133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:39.102295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf39476-9640-4c49-bf3c-6995f4638133 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:39.121892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a13713-4cd3-410e-94cf-6059e8d130e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:39.122883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a13713-4cd3-410e-94cf-6059e8d130e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:39.123358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a13713-4cd3-410e-94cf-6059e8d130e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:39.124478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a13713-4cd3-410e-94cf-6059e8d130e9 admin admin] PATH_INFO: `/v3/users/2d587668c17b4eba8b10bfa172548bca` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:39.125690 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a13713-4cd3-410e-94cf-6059e8d130e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d587668c17b4eba8b10bfa172548bca)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:39.127950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a13713-4cd3-410e-94cf-6059e8d130e9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:39.141136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1001/2011] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:38 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:29:39.143964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf39476-9640-4c49-bf3c-6995f4638133 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:39.158831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf39476-9640-4c49-bf3c-6995f4638133 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:39.175704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490bc14-5f5f-4bca-8b43-4b54c46ef67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:39.176509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490bc14-5f5f-4bca-8b43-4b54c46ef67d admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:39.176937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490bc14-5f5f-4bca-8b43-4b54c46ef67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:39.177306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490bc14-5f5f-4bca-8b43-4b54c46ef67d admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:39.178021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490bc14-5f5f-4bca-8b43-4b54c46ef67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:39.179891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490bc14-5f5f-4bca-8b43-4b54c46ef67d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:39.198179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1002/2012] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:39.201604 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-85a13713-4cd3-410e-94cf-6059e8d130e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d587668c17b4eba8b10bfa172548bca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:39.202536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490bc14-5f5f-4bca-8b43-4b54c46ef67d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:39.210153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:39.212692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-85a13713-4cd3-410e-94cf-6059e8d130e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d587668c17b4eba8b10bfa172548bca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:39.213451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-85a13713-4cd3-410e-94cf-6059e8d130e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d587668c17b4eba8b10bfa172548bca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:39.213973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490bc14-5f5f-4bca-8b43-4b54c46ef67d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:39.220763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0301dff-86a2-4e07-b648-8df2443665bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:39.221425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0301dff-86a2-4e07-b648-8df2443665bd admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:39.222577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0301dff-86a2-4e07-b648-8df2443665bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:39.222853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0301dff-86a2-4e07-b648-8df2443665bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:39.223859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0301dff-86a2-4e07-b648-8df2443665bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:39.225757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0301dff-86a2-4e07-b648-8df2443665bd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:39.240627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-85a13713-4cd3-410e-94cf-6059e8d130e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d587668c17b4eba8b10bfa172548bca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:39.250298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1003/2013] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:39.252095 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0301dff-86a2-4e07-b648-8df2443665bd admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:39.254825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e0301dff-86a2-4e07-b648-8df2443665bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:39.259548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:39.271023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-85a13713-4cd3-410e-94cf-6059e8d130e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d587668c17b4eba8b10bfa172548bca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:39.284175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0301dff-86a2-4e07-b648-8df2443665bd admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:39.285014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1011/2014] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:38 2022] DELETE /identity/v3/users/2d587668c17b4eba8b10bfa172548bca => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:29:39.291691 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dcb64-c9bb-4883-9d13-7831a8d2db4f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:39.292408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dcb64-c9bb-4883-9d13-7831a8d2db4f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:39.292938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dcb64-c9bb-4883-9d13-7831a8d2db4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:39.319468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-999dcb64-c9bb-4883-9d13-7831a8d2db4f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:39.321359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-999dcb64-c9bb-4883-9d13-7831a8d2db4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:39.417924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e8202-7635-4751-b252-6687efda40f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:39.418612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e8202-7635-4751-b252-6687efda40f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:39.419065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e8202-7635-4751-b252-6687efda40f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:39.419337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e8202-7635-4751-b252-6687efda40f2 admin admin] PATH_INFO: `/v3/projects/c954675bd66045b385b3fff72d743304/users/19d86a47e4364df291ad4c07c91621f3/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:39.455742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e8202-7635-4751-b252-6687efda40f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c954675bd66045b385b3fff72d743304, user_id=19d86a47e4364df291ad4c07c91621f3, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:39.458541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1012/2015] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:29:39.458749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e8202-7635-4751-b252-6687efda40f2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:39.461144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dcb64-c9bb-4883-9d13-7831a8d2db4f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:39.462846 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-999dcb64-c9bb-4883-9d13-7831a8d2db4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:39.490459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1004/2016] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:39 2022] PUT /identity/v3/projects/c954675bd66045b385b3fff72d743304/users/19d86a47e4364df291ad4c07c91621f3/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:39.501529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b950b6-1beb-455f-bd27-29a313d8fa42 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:39.501843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b950b6-1beb-455f-bd27-29a313d8fa42 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:39.502123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b950b6-1beb-455f-bd27-29a313d8fa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:39.584037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-01b950b6-1beb-455f-bd27-29a313d8fa42 None None] MFA Rules not processed for user `19d86a47e4364df291ad4c07c91621f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:39.586255 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-01b950b6-1beb-455f-bd27-29a313d8fa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:39.601502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dcb64-c9bb-4883-9d13-7831a8d2db4f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:39.712494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdde083-7103-40ab-bbd3-5478525a56e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:39.713220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdde083-7103-40ab-bbd3-5478525a56e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:39.713416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdde083-7103-40ab-bbd3-5478525a56e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:39.713747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdde083-7103-40ab-bbd3-5478525a56e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:39.746211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1005/2017] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:39.747778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b950b6-1beb-455f-bd27-29a313d8fa42 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:39.830305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:39.834749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdde083-7103-40ab-bbd3-5478525a56e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:39.837358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdde083-7103-40ab-bbd3-5478525a56e8 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:39.852860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1013/2018] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:39.881294 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b950b6-1beb-455f-bd27-29a313d8fa42 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:39.892920 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74f9a0-a68c-47e3-8f88-04f01120b1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:39.895553 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74f9a0-a68c-47e3-8f88-04f01120b1cd admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:39.895956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74f9a0-a68c-47e3-8f88-04f01120b1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:39.896351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74f9a0-a68c-47e3-8f88-04f01120b1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:39.898358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74f9a0-a68c-47e3-8f88-04f01120b1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:39.902795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74f9a0-a68c-47e3-8f88-04f01120b1cd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:39.925821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:39.953772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1006/2019] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:39 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:39.958140 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdde083-7103-40ab-bbd3-5478525a56e8 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:39.974450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdde083-7103-40ab-bbd3-5478525a56e8 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:40.067140 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba2ca7-9508-4264-bcd7-64873d0b349f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:40.068436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba2ca7-9508-4264-bcd7-64873d0b349f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.068818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba2ca7-9508-4264-bcd7-64873d0b349f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.069360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba2ca7-9508-4264-bcd7-64873d0b349f admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.070485 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba2ca7-9508-4264-bcd7-64873d0b349f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:40.073818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba2ca7-9508-4264-bcd7-64873d0b349f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:40.119675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1014/2020] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:39 2022] POST /identity/v3/users => generated 300 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:29:40.120635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74f9a0-a68c-47e3-8f88-04f01120b1cd admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:40.131807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74f9a0-a68c-47e3-8f88-04f01120b1cd admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:40.137706 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334e0e7-8604-47db-8f99-b51c92fbd46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:40.141432 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334e0e7-8604-47db-8f99-b51c92fbd46a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.141828 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334e0e7-8604-47db-8f99-b51c92fbd46a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.142090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334e0e7-8604-47db-8f99-b51c92fbd46a admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.142873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334e0e7-8604-47db-8f99-b51c92fbd46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:40.144616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334e0e7-8604-47db-8f99-b51c92fbd46a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:40.159696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1007/2021] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:40.160904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba2ca7-9508-4264-bcd7-64873d0b349f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:40.177491 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba2ca7-9508-4264-bcd7-64873d0b349f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:40.187347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c06c8e-a119-404b-9861-acae04b0e3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:40.188035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c06c8e-a119-404b-9861-acae04b0e3f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.188310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c06c8e-a119-404b-9861-acae04b0e3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.188588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c06c8e-a119-404b-9861-acae04b0e3f6 admin admin] PATH_INFO: `/v3/projects/630f6fce51484f799d66aa26faaf6e33/users/2ec939b9334b42dea53bf1ec077dea96/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.208683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c06c8e-a119-404b-9861-acae04b0e3f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=630f6fce51484f799d66aa26faaf6e33, user_id=2ec939b9334b42dea53bf1ec077dea96, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:40.212036 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c06c8e-a119-404b-9861-acae04b0e3f6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:40.236080 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1015/2022] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:40 2022] PUT /identity/v3/projects/630f6fce51484f799d66aa26faaf6e33/users/2ec939b9334b42dea53bf1ec077dea96/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:40.240242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334e0e7-8604-47db-8f99-b51c92fbd46a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:40.254020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334e0e7-8604-47db-8f99-b51c92fbd46a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:40.346335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ed56b-34de-45f8-977b-5be5dfe3cb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:40.347083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ed56b-34de-45f8-977b-5be5dfe3cb67 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.347799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ed56b-34de-45f8-977b-5be5dfe3cb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.348107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ed56b-34de-45f8-977b-5be5dfe3cb67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.348768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ed56b-34de-45f8-977b-5be5dfe3cb67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:40.351455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ed56b-34de-45f8-977b-5be5dfe3cb67 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:40.373499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c06c8e-a119-404b-9861-acae04b0e3f6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:40.375420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-87c06c8e-a119-404b-9861-acae04b0e3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:40.384107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1008/2023] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:40.385843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ed56b-34de-45f8-977b-5be5dfe3cb67 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:40.402406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c06c8e-a119-404b-9861-acae04b0e3f6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:40.410748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:40.412549 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.412837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.413195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd admin admin] PATH_INFO: `/v3/projects/a70ba5dbb7724b5cb0e99caf49e76f72` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.415816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a70ba5dbb7724b5cb0e99caf49e76f72)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:40.419220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:40.423068 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ed56b-34de-45f8-977b-5be5dfe3cb67 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:40.445927 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e45ed2-9c56-4b1c-9a5d-26f836f615ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:40.447826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e45ed2-9c56-4b1c-9a5d-26f836f615ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.449116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e45ed2-9c56-4b1c-9a5d-26f836f615ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.449517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e45ed2-9c56-4b1c-9a5d-26f836f615ef admin admin] PATH_INFO: `/v3/projects/630f6fce51484f799d66aa26faaf6e33/users/2ec939b9334b42dea53bf1ec077dea96/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.454640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e45ed2-9c56-4b1c-9a5d-26f836f615ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=630f6fce51484f799d66aa26faaf6e33, user_id=2ec939b9334b42dea53bf1ec077dea96, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:40.464315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e45ed2-9c56-4b1c-9a5d-26f836f615ef admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:40.490774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1009/2024] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:40 2022] PUT /identity/v3/projects/630f6fce51484f799d66aa26faaf6e33/users/2ec939b9334b42dea53bf1ec077dea96/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:40.493020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2721ed2-323d-48ae-9345-32f6a90aa260 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.493600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2721ed2-323d-48ae-9345-32f6a90aa260 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.494073 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2721ed2-323d-48ae-9345-32f6a90aa260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.523879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a70ba5dbb7724b5cb0e99caf49e76f72', 'request_id': 'req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:40.529205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a2721ed2-323d-48ae-9345-32f6a90aa260 None None] MFA Rules not processed for user `2ec939b9334b42dea53bf1ec077dea96`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:40.530766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a2721ed2-323d-48ae-9345-32f6a90aa260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:40.537510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a70ba5dbb7724b5cb0e99caf49e76f72', 'request_id': 'req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:40.546913 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a70ba5dbb7724b5cb0e99caf49e76f72', 'request_id': 'req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:40.587776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd admin admin] The token cache is being invalidate because project a70ba5dbb7724b5cb0e99caf49e76f72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:40.588333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:40.590687 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5ed0d428-1dc3-4f3a-aee3-8baf2e9577cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:40.593338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1016/2025] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:40 2022] DELETE /identity/v3/projects/a70ba5dbb7724b5cb0e99caf49e76f72 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:40.615579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a0b5b-5af7-47c7-a8bc-d9f1db4a9a7d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.615864 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a0b5b-5af7-47c7-a8bc-d9f1db4a9a7d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.616146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a0b5b-5af7-47c7-a8bc-d9f1db4a9a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.628133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1010/2026] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:29:40.635682 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2721ed2-323d-48ae-9345-32f6a90aa260 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:40.638626 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a2721ed2-323d-48ae-9345-32f6a90aa260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:40.650971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-639a0b5b-5af7-47c7-a8bc-d9f1db4a9a7d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:40.652274 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-639a0b5b-5af7-47c7-a8bc-d9f1db4a9a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:40.722658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:40.747132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:40.753245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1017/2027] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:40.763156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2721ed2-323d-48ae-9345-32f6a90aa260 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:40.763634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0473ed-21c0-4cb0-81b7-444b85ddd56d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.763921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0473ed-21c0-4cb0-81b7-444b85ddd56d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.764283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0473ed-21c0-4cb0-81b7-444b85ddd56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.767926 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2718bb-1cc0-42c9-a2ea-cb42a7130262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:40.768660 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2718bb-1cc0-42c9-a2ea-cb42a7130262 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.769006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2718bb-1cc0-42c9-a2ea-cb42a7130262 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.769368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2718bb-1cc0-42c9-a2ea-cb42a7130262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.779976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2718bb-1cc0-42c9-a2ea-cb42a7130262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:40.782679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2718bb-1cc0-42c9-a2ea-cb42a7130262 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:40.795923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1011/2028] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:40.813404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8c0473ed-21c0-4cb0-81b7-444b85ddd56d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:40.814808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8c0473ed-21c0-4cb0-81b7-444b85ddd56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:40.841786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2718bb-1cc0-42c9-a2ea-cb42a7130262 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:40.842089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1018/2029] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:29:40.846483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5d2718bb-1cc0-42c9-a2ea-cb42a7130262 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:40.846740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0473ed-21c0-4cb0-81b7-444b85ddd56d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:40.859322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0473ed-21c0-4cb0-81b7-444b85ddd56d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:40.870726 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2718bb-1cc0-42c9-a2ea-cb42a7130262 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:40.873344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642232a-8b13-48c9-9960-16dba3459550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:40.873982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642232a-8b13-48c9-9960-16dba3459550 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.874260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642232a-8b13-48c9-9960-16dba3459550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.874522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642232a-8b13-48c9-9960-16dba3459550 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.875175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642232a-8b13-48c9-9960-16dba3459550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:29:40.875471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642232a-8b13-48c9-9960-16dba3459550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:40.877242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642232a-8b13-48c9-9960-16dba3459550 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:40.878491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e16d9-24c3-45d8-8e5a-f2befc151767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:40.879548 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e16d9-24c3-45d8-8e5a-f2befc151767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.879832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e16d9-24c3-45d8-8e5a-f2befc151767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.880109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e16d9-24c3-45d8-8e5a-f2befc151767 admin admin] PATH_INFO: `/v3/users/19d86a47e4364df291ad4c07c91621f3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.881508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e16d9-24c3-45d8-8e5a-f2befc151767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d86a47e4364df291ad4c07c91621f3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:40.883930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e16d9-24c3-45d8-8e5a-f2befc151767 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:40.908602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1019/2030] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:29:40.911471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642232a-8b13-48c9-9960-16dba3459550 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:40.932962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642232a-8b13-48c9-9960-16dba3459550 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:40.938847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce41b68-4356-41e4-9352-ef96030961df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:40.939692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce41b68-4356-41e4-9352-ef96030961df admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:40.940235 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce41b68-4356-41e4-9352-ef96030961df admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:40.940648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce41b68-4356-41e4-9352-ef96030961df admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:40.941514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce41b68-4356-41e4-9352-ef96030961df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:40.944483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce41b68-4356-41e4-9352-ef96030961df admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:40.982172 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d9e16d9-24c3-45d8-8e5a-f2befc151767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d86a47e4364df291ad4c07c91621f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:40.999622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d9e16d9-24c3-45d8-8e5a-f2befc151767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d86a47e4364df291ad4c07c91621f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:41.000418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d9e16d9-24c3-45d8-8e5a-f2befc151767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d86a47e4364df291ad4c07c91621f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:41.001838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1020/2031] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:40 2022] POST /identity/v3/projects => generated 369 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:41.006779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce41b68-4356-41e4-9352-ef96030961df admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:41.010692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6ce41b68-4356-41e4-9352-ef96030961df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:41.020186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d9e16d9-24c3-45d8-8e5a-f2befc151767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d86a47e4364df291ad4c07c91621f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:41.044778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d9e16d9-24c3-45d8-8e5a-f2befc151767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d86a47e4364df291ad4c07c91621f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:41.056651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1012/2032] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:40 2022] DELETE /identity/v3/users/19d86a47e4364df291ad4c07c91621f3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:41.058879 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a32815-0126-4f01-9f18-97071da4a121 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:41.059750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a32815-0126-4f01-9f18-97071da4a121 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:41.060029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a32815-0126-4f01-9f18-97071da4a121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:41.092393 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d6a32815-0126-4f01-9f18-97071da4a121 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:41.094903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d6a32815-0126-4f01-9f18-97071da4a121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:41.160332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce41b68-4356-41e4-9352-ef96030961df admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:41.279990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1013/2033] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:41.295172 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a32815-0126-4f01-9f18-97071da4a121 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:41.296600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d6a32815-0126-4f01-9f18-97071da4a121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:41.299499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4a515-c765-4e2c-8425-a0ce5c9afcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:41.300298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4a515-c765-4e2c-8425-a0ce5c9afcc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:41.300650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4a515-c765-4e2c-8425-a0ce5c9afcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:41.300846 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4a515-c765-4e2c-8425-a0ce5c9afcc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:41.301609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4a515-c765-4e2c-8425-a0ce5c9afcc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:41.304149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4a515-c765-4e2c-8425-a0ce5c9afcc2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:41.348672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1021/2034] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:41 2022] POST /identity/v3/users => generated 285 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:29:41.349925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4a515-c765-4e2c-8425-a0ce5c9afcc2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:41.362850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4a515-c765-4e2c-8425-a0ce5c9afcc2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:41.369859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ca0dd-ed99-4cb3-adb0-aac581b26f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:41.370988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ca0dd-ed99-4cb3-adb0-aac581b26f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:41.371442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ca0dd-ed99-4cb3-adb0-aac581b26f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:41.372032 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ca0dd-ed99-4cb3-adb0-aac581b26f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:41.372966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ca0dd-ed99-4cb3-adb0-aac581b26f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:41.375568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ca0dd-ed99-4cb3-adb0-aac581b26f52 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:41.391403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a32815-0126-4f01-9f18-97071da4a121 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:41.392470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1022/2035] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:41.393719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ca0dd-ed99-4cb3-adb0-aac581b26f52 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:41.402482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d38bcf-1c2b-407a-b5e7-eaf96cfd0888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:41.405606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d38bcf-1c2b-407a-b5e7-eaf96cfd0888 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:41.405884 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d38bcf-1c2b-407a-b5e7-eaf96cfd0888 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:41.406157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d38bcf-1c2b-407a-b5e7-eaf96cfd0888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:41.408317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ca0dd-ed99-4cb3-adb0-aac581b26f52 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:41.415094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e7360-e072-4059-8b23-f2d2da114298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:41.415943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e7360-e072-4059-8b23-f2d2da114298 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:41.416228 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e7360-e072-4059-8b23-f2d2da114298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:41.416527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e7360-e072-4059-8b23-f2d2da114298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:41.417209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e7360-e072-4059-8b23-f2d2da114298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:41.419141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e7360-e072-4059-8b23-f2d2da114298 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:41.422162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d38bcf-1c2b-407a-b5e7-eaf96cfd0888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:41.424071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d38bcf-1c2b-407a-b5e7-eaf96cfd0888 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:41.437794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1014/2036] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:29:41.439559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1023/2037] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:41.443564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d38bcf-1c2b-407a-b5e7-eaf96cfd0888 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:41.453934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d38bcf-1c2b-407a-b5e7-eaf96cfd0888 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:41.460523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162b1b2-414e-4f42-b622-92ad89bf9b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:41.461339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162b1b2-414e-4f42-b622-92ad89bf9b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:41.461695 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162b1b2-414e-4f42-b622-92ad89bf9b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:41.462023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162b1b2-414e-4f42-b622-92ad89bf9b1a admin admin] PATH_INFO: `/v3/projects/097e46a08d19498aadf0ef0cd38c3cf7/users/923d21322c084edcb323f7b431d0f17d/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:41.464656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:41.481429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162b1b2-414e-4f42-b622-92ad89bf9b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=097e46a08d19498aadf0ef0cd38c3cf7, user_id=923d21322c084edcb323f7b431d0f17d, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:41.484191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162b1b2-414e-4f42-b622-92ad89bf9b1a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:41.522118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1015/2038] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:41 2022] PUT /identity/v3/projects/097e46a08d19498aadf0ef0cd38c3cf7/users/923d21322c084edcb323f7b431d0f17d/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:41.532864 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219d978-cc63-454e-b080-abffd9ab8d9c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:41.533250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219d978-cc63-454e-b080-abffd9ab8d9c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:41.533527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219d978-cc63-454e-b080-abffd9ab8d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:41.630520 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9219d978-cc63-454e-b080-abffd9ab8d9c None None] MFA Rules not processed for user `923d21322c084edcb323f7b431d0f17d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:41.633686 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9219d978-cc63-454e-b080-abffd9ab8d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:41.647103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:41.694115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162b1b2-414e-4f42-b622-92ad89bf9b1a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:41.695914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c162b1b2-414e-4f42-b622-92ad89bf9b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:41.761456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1024/2039] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:41.775529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219d978-cc63-454e-b080-abffd9ab8d9c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:41.778387 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9219d978-cc63-454e-b080-abffd9ab8d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:41.795491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162b1b2-414e-4f42-b622-92ad89bf9b1a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:41.801701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:41.802459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:41.802810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:41.803778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] PATH_INFO: `/v3/users/e6e77c183c7d459fb8042d4455477f7f` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:41.805071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e77c183c7d459fb8042d4455477f7f)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:41.807112 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:41.900959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e77c183c7d459fb8042d4455477f7f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:41.918653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e77c183c7d459fb8042d4455477f7f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:41.921543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e77c183c7d459fb8042d4455477f7f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:41.941733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e77c183c7d459fb8042d4455477f7f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:41.980527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e77c183c7d459fb8042d4455477f7f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:41.996262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1016/2040] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:41 2022] DELETE /identity/v3/users/e6e77c183c7d459fb8042d4455477f7f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:29:41.998036 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219d978-cc63-454e-b080-abffd9ab8d9c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:42.061908 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05612880-5eb9-4f4e-ab20-95e4cf74a611 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf8a0ae03c94aa78cc2ab0e4e1c87b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:42.062601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05612880-5eb9-4f4e-ab20-95e4cf74a611 service glance] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:42.062882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05612880-5eb9-4f4e-ab20-95e4cf74a611 service glance] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:42.063153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05612880-5eb9-4f4e-ab20-95e4cf74a611 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:42.066026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-05612880-5eb9-4f4e-ab20-95e4cf74a611 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:42.106704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:42.108242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:42.185923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05612880-5eb9-4f4e-ab20-95e4cf74a611 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:42.187739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05612880-5eb9-4f4e-ab20-95e4cf74a611 service glance] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:42.193366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:42.205671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1025/2041] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:42.222507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d696a-0751-420a-9cfa-4b8e419a91ef admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:42.232567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:42.233265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:42.233544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:42.233845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] PATH_INFO: `/v3/projects/c954675bd66045b385b3fff72d743304` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:42.236864 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c954675bd66045b385b3fff72d743304)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:42.239598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:42.243191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:42.311630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c954675bd66045b385b3fff72d743304', 'request_id': 'req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.323668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c954675bd66045b385b3fff72d743304', 'request_id': 'req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.333784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c954675bd66045b385b3fff72d743304', 'request_id': 'req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.358892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2040e9b-146a-4375-9f67-025c34bf6183 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:42.359219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2040e9b-146a-4375-9f67-025c34bf6183 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:42.359537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2040e9b-146a-4375-9f67-025c34bf6183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:42.363445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] The token cache is being invalidate because project c954675bd66045b385b3fff72d743304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:42.365938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.366640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.373508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1017/2042] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:42 2022] DELETE /identity/v3/projects/c954675bd66045b385b3fff72d743304 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:42.377667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:42.390406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:42.447813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f2040e9b-146a-4375-9f67-025c34bf6183 None None] MFA Rules not processed for user `f6a5aff543234d7a9976c2e43453aef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:42.449166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f2040e9b-146a-4375-9f67-025c34bf6183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:42.541523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577d0e1-c0df-4fb3-b95e-fa7627bbfc07 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:42.548069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:42.548872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:42.549250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:42.549627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] PATH_INFO: `/v3/users/2ec939b9334b42dea53bf1ec077dea96` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:42.551039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec939b9334b42dea53bf1ec077dea96)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:42.553915 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:42.585338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1026/2043] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:42.586334 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2040e9b-146a-4375-9f67-025c34bf6183 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:42.590783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f2040e9b-146a-4375-9f67-025c34bf6183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:42.609466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2040e9b-146a-4375-9f67-025c34bf6183 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:42.615515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:42.616270 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:42.616529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:42.616799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] PATH_INFO: `/v3/projects/518ea7ac8ba9429688cceb5d40067fb5` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:42.618066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=518ea7ac8ba9429688cceb5d40067fb5)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:42.620474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:42.637073 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec939b9334b42dea53bf1ec077dea96'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.663171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec939b9334b42dea53bf1ec077dea96'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.663952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec939b9334b42dea53bf1ec077dea96'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.683971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec939b9334b42dea53bf1ec077dea96'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.710232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec939b9334b42dea53bf1ec077dea96'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.722605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '518ea7ac8ba9429688cceb5d40067fb5', 'request_id': 'req-12ba3c61-47f2-4a05-8347-143f07f47918'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.731752 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '518ea7ac8ba9429688cceb5d40067fb5', 'request_id': 'req-12ba3c61-47f2-4a05-8347-143f07f47918'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.732873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1018/2044] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:42 2022] DELETE /identity/v3/users/2ec939b9334b42dea53bf1ec077dea96 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:42.735412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:42.742810 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '518ea7ac8ba9429688cceb5d40067fb5', 'request_id': 'req-12ba3c61-47f2-4a05-8347-143f07f47918'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.756674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] The token cache is being invalidate because project 518ea7ac8ba9429688cceb5d40067fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:42.756969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.757707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:42.759103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:42.759337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1027/2045] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:42 2022] DELETE /identity/v3/projects/518ea7ac8ba9429688cceb5d40067fb5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:42.763712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:42.765193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:42.781657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:42.858051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee4e5d-85ce-4848-ab1f-ad20e69a793d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:42.864613 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dce1f-a01a-4297-84e2-7de6f572a52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:42.865592 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dce1f-a01a-4297-84e2-7de6f572a52f admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:42.865812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dce1f-a01a-4297-84e2-7de6f572a52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:42.866084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dce1f-a01a-4297-84e2-7de6f572a52f admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:42.866732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dce1f-a01a-4297-84e2-7de6f572a52f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:42.869091 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dce1f-a01a-4297-84e2-7de6f572a52f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:42.919148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba3c61-47f2-4a05-8347-143f07f47918 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:42.934301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:42.936581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1019/2046] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:42 2022] POST /identity/v3/projects => generated 385 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:42.936856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:42.937161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:42.937444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] PATH_INFO: `/v3/users/00cee67e83df4307809596fb2af2abee` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:42.940654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dce1f-a01a-4297-84e2-7de6f572a52f admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:42.941816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] RBAC: Authorizing `identity:delete_user(user_id=00cee67e83df4307809596fb2af2abee)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:42.942339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d67dce1f-a01a-4297-84e2-7de6f572a52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:42.944220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:42.982482 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:43.057585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00cee67e83df4307809596fb2af2abee'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.080229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00cee67e83df4307809596fb2af2abee'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.081392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00cee67e83df4307809596fb2af2abee'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.096077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00cee67e83df4307809596fb2af2abee'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.110204 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00cee67e83df4307809596fb2af2abee'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.118679 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1028/2047] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:42 2022] DELETE /identity/v3/users/00cee67e83df4307809596fb2af2abee => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:43.119129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dce1f-a01a-4297-84e2-7de6f572a52f admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:43.190643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a306cf-107a-43fe-9446-c9bf7163e28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:43.191273 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a306cf-107a-43fe-9446-c9bf7163e28a admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.192160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a306cf-107a-43fe-9446-c9bf7163e28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.192490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a306cf-107a-43fe-9446-c9bf7163e28a admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.193172 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a306cf-107a-43fe-9446-c9bf7163e28a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:43.195353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a306cf-107a-43fe-9446-c9bf7163e28a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:43.248857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1020/2048] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:42 2022] POST /identity/v3/users => generated 299 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:43.249272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:43.250936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:43.283444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a306cf-107a-43fe-9446-c9bf7163e28a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:43.287395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-44a306cf-107a-43fe-9446-c9bf7163e28a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:43.299380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902b1ef-8140-40a9-a165-16e8d401998b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:43.306766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a306cf-107a-43fe-9446-c9bf7163e28a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:43.312520 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:43.313290 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.313555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.313787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] PATH_INFO: `/v3/projects/630f6fce51484f799d66aa26faaf6e33` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.315129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=630f6fce51484f799d66aa26faaf6e33)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:43.317158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:43.340512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e13c4c-9210-4708-adc2-ad27416ebd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:43.341333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e13c4c-9210-4708-adc2-ad27416ebd38 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.341618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e13c4c-9210-4708-adc2-ad27416ebd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.341889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e13c4c-9210-4708-adc2-ad27416ebd38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.342847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e13c4c-9210-4708-adc2-ad27416ebd38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:43.345705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e13c4c-9210-4708-adc2-ad27416ebd38 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:43.362661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1029/2049] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:43.363710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e13c4c-9210-4708-adc2-ad27416ebd38 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:43.365922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-91e13c4c-9210-4708-adc2-ad27416ebd38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:43.376370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:43.401049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e13c4c-9210-4708-adc2-ad27416ebd38 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:43.405746 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '630f6fce51484f799d66aa26faaf6e33', 'request_id': 'req-3cb093f3-b991-449c-aa83-77a396d1a5fd'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.422680 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '630f6fce51484f799d66aa26faaf6e33', 'request_id': 'req-3cb093f3-b991-449c-aa83-77a396d1a5fd'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.431212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '630f6fce51484f799d66aa26faaf6e33', 'request_id': 'req-3cb093f3-b991-449c-aa83-77a396d1a5fd'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.444864 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] The token cache is being invalidate because project 630f6fce51484f799d66aa26faaf6e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:43.445517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.446289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.451517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1021/2050] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:43 2022] DELETE /identity/v3/projects/630f6fce51484f799d66aa26faaf6e33 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:43.451517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:43.453124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:43.496646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:43.497446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.497779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.498120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] PATH_INFO: `/v3/projects/84e45a668f214e35be48d9782e5a4042` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.499355 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e45a668f214e35be48d9782e5a4042)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:43.501632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:43.540980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb093f3-b991-449c-aa83-77a396d1a5fd admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:43.546882 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93c003-0113-47aa-a090-892bafee8d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:43.549046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93c003-0113-47aa-a090-892bafee8d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.550570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93c003-0113-47aa-a090-892bafee8d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.550844 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93c003-0113-47aa-a090-892bafee8d34 admin admin] PATH_INFO: `/v3/projects/191e745d2d9245da8d2df2e4f9a7814a/users/8606d928811c4c64aea5db0c71a81c04/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.561781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e45a668f214e35be48d9782e5a4042', 'request_id': 'req-bd93e945-e88d-40ac-9ad1-056763b7008b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.567574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93c003-0113-47aa-a090-892bafee8d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=191e745d2d9245da8d2df2e4f9a7814a, user_id=8606d928811c4c64aea5db0c71a81c04, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:43.570540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93c003-0113-47aa-a090-892bafee8d34 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:43.575260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e45a668f214e35be48d9782e5a4042', 'request_id': 'req-bd93e945-e88d-40ac-9ad1-056763b7008b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.584101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e45a668f214e35be48d9782e5a4042', 'request_id': 'req-bd93e945-e88d-40ac-9ad1-056763b7008b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.589119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1022/2051] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:43 2022] PUT /identity/v3/projects/191e745d2d9245da8d2df2e4f9a7814a/users/8606d928811c4c64aea5db0c71a81c04/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:43.590499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65482261-1d85-4e3a-9ca8-80f0372e8c99 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.590853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65482261-1d85-4e3a-9ca8-80f0372e8c99 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.591200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65482261-1d85-4e3a-9ca8-80f0372e8c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.600261 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] The token cache is being invalidate because project 84e45a668f214e35be48d9782e5a4042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:43.600550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.601615 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:43.605359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1030/2052] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:43 2022] DELETE /identity/v3/projects/84e45a668f214e35be48d9782e5a4042 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:43.605359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:43.605359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:43.615902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:43.617970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-65482261-1d85-4e3a-9ca8-80f0372e8c99 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:43.619245 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-65482261-1d85-4e3a-9ca8-80f0372e8c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:43.693274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1023/2053] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:43.693532 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93e945-e88d-40ac-9ad1-056763b7008b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:43.695569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346bd3a-c697-418a-8c4c-36019cab267b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.696008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346bd3a-c697-418a-8c4c-36019cab267b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.696996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346bd3a-c697-418a-8c4c-36019cab267b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.705769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fe881-7a4b-4324-8d12-5ca82873461f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:43.706503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fe881-7a4b-4324-8d12-5ca82873461f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.706839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fe881-7a4b-4324-8d12-5ca82873461f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.707405 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fe881-7a4b-4324-8d12-5ca82873461f admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.708020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885fe881-7a4b-4324-8d12-5ca82873461f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:43.709800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885fe881-7a4b-4324-8d12-5ca82873461f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:43.719893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b346bd3a-c697-418a-8c4c-36019cab267b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:43.721692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b346bd3a-c697-418a-8c4c-36019cab267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:43.722871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1031/2054] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:43.724103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee0a55-e970-41e6-8eca-d60176d1a9eb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.725232 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee0a55-e970-41e6-8eca-d60176d1a9eb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.725564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee0a55-e970-41e6-8eca-d60176d1a9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.731887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1024/2055] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:43.733140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346bd3a-c697-418a-8c4c-36019cab267b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:43.752853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346bd3a-c697-418a-8c4c-36019cab267b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:43.755645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0dee0a55-e970-41e6-8eca-d60176d1a9eb None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:43.756932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0dee0a55-e970-41e6-8eca-d60176d1a9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:43.759258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8549309-9443-4ac4-b982-076ed606506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:43.760023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8549309-9443-4ac4-b982-076ed606506f admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.760395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8549309-9443-4ac4-b982-076ed606506f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.760699 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8549309-9443-4ac4-b982-076ed606506f admin admin] PATH_INFO: `/v3/projects/191e745d2d9245da8d2df2e4f9a7814a/users/8606d928811c4c64aea5db0c71a81c04/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.763523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8549309-9443-4ac4-b982-076ed606506f admin admin] RBAC: Authorizing `identity:create_grant(project_id=191e745d2d9245da8d2df2e4f9a7814a, user_id=8606d928811c4c64aea5db0c71a81c04, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:43.765339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1032/2056] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:43.766380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8549309-9443-4ac4-b982-076ed606506f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:43.770635 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f78c2-379e-4879-b4c2-7c2dec8957b0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.771086 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f78c2-379e-4879-b4c2-7c2dec8957b0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.771504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f78c2-379e-4879-b4c2-7c2dec8957b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.781530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1025/2057] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:43 2022] PUT /identity/v3/projects/191e745d2d9245da8d2df2e4f9a7814a/users/8606d928811c4c64aea5db0c71a81c04/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:43.781530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8549309-9443-4ac4-b982-076ed606506f admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:43.799194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d54f78c2-379e-4879-b4c2-7c2dec8957b0 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:43.800417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d54f78c2-379e-4879-b4c2-7c2dec8957b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:43.820216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:43.888186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8549309-9443-4ac4-b982-076ed606506f admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:43.890068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1033/2058] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:43.894037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f78c2-379e-4879-b4c2-7c2dec8957b0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:43.903173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b338a-ffaa-4014-8e9c-053b048c155a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:43.903732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b338a-ffaa-4014-8e9c-053b048c155a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.904008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b338a-ffaa-4014-8e9c-053b048c155a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.904276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b338a-ffaa-4014-8e9c-053b048c155a admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.904979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b338a-ffaa-4014-8e9c-053b048c155a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:29:43.905421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b338a-ffaa-4014-8e9c-053b048c155a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:43.905789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f78c2-379e-4879-b4c2-7c2dec8957b0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:43.907040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b338a-ffaa-4014-8e9c-053b048c155a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:43.914528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ad834-bd9c-476d-bd07-a423fedb54ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:43.915182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ad834-bd9c-476d-bd07-a423fedb54ef admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.917021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ad834-bd9c-476d-bd07-a423fedb54ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.917336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ad834-bd9c-476d-bd07-a423fedb54ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.918067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ad834-bd9c-476d-bd07-a423fedb54ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:29:43.918405 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ad834-bd9c-476d-bd07-a423fedb54ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:43.920169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ad834-bd9c-476d-bd07-a423fedb54ef admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:43.937241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1034/2059] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:29:43.938809 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ad834-bd9c-476d-bd07-a423fedb54ef admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:43.939868 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1026/2060] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:29:43.946916 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b338a-ffaa-4014-8e9c-053b048c155a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:43.956539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ad834-bd9c-476d-bd07-a423fedb54ef admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:43.964266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c242f38-299f-4953-800f-5258438b9024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:43.965031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c242f38-299f-4953-800f-5258438b9024 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.965501 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c242f38-299f-4953-800f-5258438b9024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.966788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c242f38-299f-4953-800f-5258438b9024 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.967702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c242f38-299f-4953-800f-5258438b9024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:43.971759 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c242f38-299f-4953-800f-5258438b9024 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:43.974336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b338a-ffaa-4014-8e9c-053b048c155a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:43.987203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a02c0-cdce-4f3b-8fe2-25cbcdbe3746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:43.988395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a02c0-cdce-4f3b-8fe2-25cbcdbe3746 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:43.989819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a02c0-cdce-4f3b-8fe2-25cbcdbe3746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:43.990099 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a02c0-cdce-4f3b-8fe2-25cbcdbe3746 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:43.990753 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a02c0-cdce-4f3b-8fe2-25cbcdbe3746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.006595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a02c0-cdce-4f3b-8fe2-25cbcdbe3746 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.026860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1035/2061] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:43 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:44.028272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c242f38-299f-4953-800f-5258438b9024 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.063786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1027/2062] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:43 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:44.067998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a02c0-cdce-4f3b-8fe2-25cbcdbe3746 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.163393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c242f38-299f-4953-800f-5258438b9024 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:44.172713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:44.197847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a02c0-cdce-4f3b-8fe2-25cbcdbe3746 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:44.205228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d5b15-61ba-4a58-9d04-78175c93ed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:44.206100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d5b15-61ba-4a58-9d04-78175c93ed51 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.206613 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d5b15-61ba-4a58-9d04-78175c93ed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.207078 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d5b15-61ba-4a58-9d04-78175c93ed51 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.207930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d5b15-61ba-4a58-9d04-78175c93ed51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.210443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d5b15-61ba-4a58-9d04-78175c93ed51 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.244559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1028/2063] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:44 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:44.250133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d5b15-61ba-4a58-9d04-78175c93ed51 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.278544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d5b15-61ba-4a58-9d04-78175c93ed51 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:44.285134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ac472-fe3f-4f79-8170-74e4ca9d2cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:44.285920 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f03063-6667-4b61-a052-02ffeff3d143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:44.286259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ac472-fe3f-4f79-8170-74e4ca9d2cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.286542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ac472-fe3f-4f79-8170-74e4ca9d2cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.286868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ac472-fe3f-4f79-8170-74e4ca9d2cf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.287062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f03063-6667-4b61-a052-02ffeff3d143 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.287481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f03063-6667-4b61-a052-02ffeff3d143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.287584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ac472-fe3f-4f79-8170-74e4ca9d2cf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.287984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f03063-6667-4b61-a052-02ffeff3d143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.288712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f03063-6667-4b61-a052-02ffeff3d143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.291251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f03063-6667-4b61-a052-02ffeff3d143 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.291774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ac472-fe3f-4f79-8170-74e4ca9d2cf8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.305921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1029/2064] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:44.309937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f03063-6667-4b61-a052-02ffeff3d143 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.326177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f03063-6667-4b61-a052-02ffeff3d143 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:44.334960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb664dc9-f09b-4fb0-89a3-125c5272fa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:44.335747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb664dc9-f09b-4fb0-89a3-125c5272fa30 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.336260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb664dc9-f09b-4fb0-89a3-125c5272fa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.336744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb664dc9-f09b-4fb0-89a3-125c5272fa30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.337683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb664dc9-f09b-4fb0-89a3-125c5272fa30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.339602 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb664dc9-f09b-4fb0-89a3-125c5272fa30 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.342200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1036/2065] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:44 2022] POST /identity/v3/users => generated 288 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:29:44.343558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ac472-fe3f-4f79-8170-74e4ca9d2cf8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.354802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1030/2066] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:29:44.362559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb664dc9-f09b-4fb0-89a3-125c5272fa30 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.376868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ac472-fe3f-4f79-8170-74e4ca9d2cf8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:44.386017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8b782-42de-4d1e-a675-8248a1439905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:44.389923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb664dc9-f09b-4fb0-89a3-125c5272fa30 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:44.390448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8b782-42de-4d1e-a675-8248a1439905 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.390776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8b782-42de-4d1e-a675-8248a1439905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.391049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8b782-42de-4d1e-a675-8248a1439905 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.392898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8b782-42de-4d1e-a675-8248a1439905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.396355 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8b782-42de-4d1e-a675-8248a1439905 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.401876 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fdcb0-43e8-4027-ba4a-48aacf90ba7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:44.404943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fdcb0-43e8-4027-ba4a-48aacf90ba7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.404943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fdcb0-43e8-4027-ba4a-48aacf90ba7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.405451 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fdcb0-43e8-4027-ba4a-48aacf90ba7f admin admin] PATH_INFO: `/v3/projects/d4c2523a071e49938bea216b99d68bac/users/d655cf758307466a9d58b1f16a7a6cbb/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.422814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1037/2067] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:44.427652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8b782-42de-4d1e-a675-8248a1439905 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.447706 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fdcb0-43e8-4027-ba4a-48aacf90ba7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4c2523a071e49938bea216b99d68bac, user_id=d655cf758307466a9d58b1f16a7a6cbb, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.453646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8b782-42de-4d1e-a675-8248a1439905 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:44.455161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fdcb0-43e8-4027-ba4a-48aacf90ba7f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.468471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6a963-d3be-44df-9a00-7657fb8c264b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:44.469212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6a963-d3be-44df-9a00-7657fb8c264b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.470426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6a963-d3be-44df-9a00-7657fb8c264b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.470713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6a963-d3be-44df-9a00-7657fb8c264b admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.472246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6a963-d3be-44df-9a00-7657fb8c264b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.474010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6a963-d3be-44df-9a00-7657fb8c264b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.488053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1031/2068] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:44 2022] PUT /identity/v3/projects/d4c2523a071e49938bea216b99d68bac/users/d655cf758307466a9d58b1f16a7a6cbb/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:44.499592 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef47f0-6196-4d88-94b1-33cfaa01e32b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.499876 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef47f0-6196-4d88-94b1-33cfaa01e32b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.500183 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef47f0-6196-4d88-94b1-33cfaa01e32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.507437 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1038/2069] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:44.511616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6a963-d3be-44df-9a00-7657fb8c264b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.533387 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6a963-d3be-44df-9a00-7657fb8c264b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:44.540850 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-aeef47f0-6196-4d88-94b1-33cfaa01e32b None None] MFA Rules not processed for user `d655cf758307466a9d58b1f16a7a6cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:44.542936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-aeef47f0-6196-4d88-94b1-33cfaa01e32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:44.609094 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:44.654483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca57108-459b-4ad4-919d-79c6d9de553f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:44.655185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca57108-459b-4ad4-919d-79c6d9de553f admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.655469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca57108-459b-4ad4-919d-79c6d9de553f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.655880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca57108-459b-4ad4-919d-79c6d9de553f admin admin] PATH_INFO: `/v3/projects/0782cfa4b62c4dc1a1fc9c241e0cb606/users/893046830f204e00966b35a8e223c9ce/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.673565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1032/2070] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:44.674736 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef47f0-6196-4d88-94b1-33cfaa01e32b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.694889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef47f0-6196-4d88-94b1-33cfaa01e32b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:44.696377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca57108-459b-4ad4-919d-79c6d9de553f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0782cfa4b62c4dc1a1fc9c241e0cb606, user_id=893046830f204e00966b35a8e223c9ce, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.699521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca57108-459b-4ad4-919d-79c6d9de553f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.700484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e2be9-8d24-49d2-a0ea-5f2d70dadbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:44.701308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e2be9-8d24-49d2-a0ea-5f2d70dadbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.702238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e2be9-8d24-49d2-a0ea-5f2d70dadbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.702537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e2be9-8d24-49d2-a0ea-5f2d70dadbd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.704412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e2be9-8d24-49d2-a0ea-5f2d70dadbd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.706667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e2be9-8d24-49d2-a0ea-5f2d70dadbd8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.729859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1039/2071] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:44 2022] PUT /identity/v3/projects/0782cfa4b62c4dc1a1fc9c241e0cb606/users/893046830f204e00966b35a8e223c9ce/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:29:44.731579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90d275-0377-4039-b773-d948248d6f5d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.731905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90d275-0377-4039-b773-d948248d6f5d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.732244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90d275-0377-4039-b773-d948248d6f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.761614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1033/2072] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:44 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:44.765883 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e2be9-8d24-49d2-a0ea-5f2d70dadbd8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.768258 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dd90d275-0377-4039-b773-d948248d6f5d None None] MFA Rules not processed for user `893046830f204e00966b35a8e223c9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:44.769645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dd90d275-0377-4039-b773-d948248d6f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:44.876268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:44.880261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e2be9-8d24-49d2-a0ea-5f2d70dadbd8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:44.887472 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2677164-4f51-477f-8489-d7925522607c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:44.889019 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2677164-4f51-477f-8489-d7925522607c admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.889594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2677164-4f51-477f-8489-d7925522607c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.890746 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2677164-4f51-477f-8489-d7925522607c admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.891841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2677164-4f51-477f-8489-d7925522607c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.894873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2677164-4f51-477f-8489-d7925522607c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.917577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1040/2073] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:44.922620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90d275-0377-4039-b773-d948248d6f5d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.928980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1034/2074] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:44 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:44.932778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2677164-4f51-477f-8489-d7925522607c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.936654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90d275-0377-4039-b773-d948248d6f5d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:44.942510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71b67d-1053-4881-ae3e-90f179a123c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:44.943207 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71b67d-1053-4881-ae3e-90f179a123c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.943576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71b67d-1053-4881-ae3e-90f179a123c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.944219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71b67d-1053-4881-ae3e-90f179a123c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.944603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71b67d-1053-4881-ae3e-90f179a123c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.946589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2677164-4f51-477f-8489-d7925522607c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:44.947079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71b67d-1053-4881-ae3e-90f179a123c1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.955400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c5aa2-edac-4cb1-a9d7-36648ef076f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:44.956675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c5aa2-edac-4cb1-a9d7-36648ef076f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:44.956675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c5aa2-edac-4cb1-a9d7-36648ef076f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:44.956887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c5aa2-edac-4cb1-a9d7-36648ef076f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:44.957762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c5aa2-edac-4cb1-a9d7-36648ef076f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:44.960181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c5aa2-edac-4cb1-a9d7-36648ef076f1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:44.975618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1035/2075] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:44.979810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c5aa2-edac-4cb1-a9d7-36648ef076f1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.985728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1041/2076] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:44 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:44.989996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71b67d-1053-4881-ae3e-90f179a123c1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:44.992773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c5aa2-edac-4cb1-a9d7-36648ef076f1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:45.069900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71b67d-1053-4881-ae3e-90f179a123c1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:45.081160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658716d4-3dcb-44fc-97ca-eea9c9a24684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:45.081796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658716d4-3dcb-44fc-97ca-eea9c9a24684 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.082073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658716d4-3dcb-44fc-97ca-eea9c9a24684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.082351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658716d4-3dcb-44fc-97ca-eea9c9a24684 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.082990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658716d4-3dcb-44fc-97ca-eea9c9a24684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:45.085765 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658716d4-3dcb-44fc-97ca-eea9c9a24684 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:45.112347 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47a03e-191f-48b9-b638-fe2ada22b9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:45.113302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47a03e-191f-48b9-b638-fe2ada22b9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.113678 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47a03e-191f-48b9-b638-fe2ada22b9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.114019 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47a03e-191f-48b9-b638-fe2ada22b9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.114871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47a03e-191f-48b9-b638-fe2ada22b9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:45.116776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1042/2077] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:44 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:45.120525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658716d4-3dcb-44fc-97ca-eea9c9a24684 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:45.121375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47a03e-191f-48b9-b638-fe2ada22b9c6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:45.134436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658716d4-3dcb-44fc-97ca-eea9c9a24684 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:45.134649 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1036/2078] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:45.140974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7abb5-b775-46d8-91b3-f5b9aeb399b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:45.141803 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7abb5-b775-46d8-91b3-f5b9aeb399b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.142228 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7abb5-b775-46d8-91b3-f5b9aeb399b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.142693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7abb5-b775-46d8-91b3-f5b9aeb399b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.143514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7abb5-b775-46d8-91b3-f5b9aeb399b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:45.145458 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7abb5-b775-46d8-91b3-f5b9aeb399b5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:45.151656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47a03e-191f-48b9-b638-fe2ada22b9c6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:45.160603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1043/2079] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:29:45.165320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7abb5-b775-46d8-91b3-f5b9aeb399b5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:45.179843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7abb5-b775-46d8-91b3-f5b9aeb399b5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:45.182506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47a03e-191f-48b9-b638-fe2ada22b9c6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:45.185281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e248e4b-16ee-4d90-a134-91c1599c91c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:45.186171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e248e4b-16ee-4d90-a134-91c1599c91c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.187217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e248e4b-16ee-4d90-a134-91c1599c91c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.187217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e248e4b-16ee-4d90-a134-91c1599c91c6 admin admin] PATH_INFO: `/v3/projects/78948d9a0aff492ba4dc0657e1730467/users/ebdc6beeb54d426cb5956675146299e2/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.195698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34c8e5-3403-493f-84b7-cb5ce7c2020f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:45.198700 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34c8e5-3403-493f-84b7-cb5ce7c2020f admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.199370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34c8e5-3403-493f-84b7-cb5ce7c2020f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.200702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34c8e5-3403-493f-84b7-cb5ce7c2020f admin admin] PATH_INFO: `/v3/projects/a553772c3e474fcd80de0d3a3b514dfd/users/e17323fdffde4e75a1e1fa7757822528/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.209627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e248e4b-16ee-4d90-a134-91c1599c91c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78948d9a0aff492ba4dc0657e1730467, user_id=ebdc6beeb54d426cb5956675146299e2, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:45.214252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e248e4b-16ee-4d90-a134-91c1599c91c6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:45.236956 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1044/2080] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:45 2022] PUT /identity/v3/projects/78948d9a0aff492ba4dc0657e1730467/users/ebdc6beeb54d426cb5956675146299e2/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:45.252295 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e248e4b-16ee-4d90-a134-91c1599c91c6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:45.263247 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34c8e5-3403-493f-84b7-cb5ce7c2020f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a553772c3e474fcd80de0d3a3b514dfd, user_id=e17323fdffde4e75a1e1fa7757822528, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:45.267816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34c8e5-3403-493f-84b7-cb5ce7c2020f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:45.272240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:45.304808 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1037/2081] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:45 2022] PUT /identity/v3/projects/a553772c3e474fcd80de0d3a3b514dfd/users/e17323fdffde4e75a1e1fa7757822528/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:45.319343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fc9cc-901c-4450-bb73-84c50d1073a5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.319757 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fc9cc-901c-4450-bb73-84c50d1073a5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.320102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fc9cc-901c-4450-bb73-84c50d1073a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.363729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e248e4b-16ee-4d90-a134-91c1599c91c6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:45.391000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-878fc9cc-901c-4450-bb73-84c50d1073a5 None None] MFA Rules not processed for user `e17323fdffde4e75a1e1fa7757822528`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:45.395317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-878fc9cc-901c-4450-bb73-84c50d1073a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:45.412984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:45.475075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020f13e-1794-4967-888f-35b782d3b9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:45.476559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020f13e-1794-4967-888f-35b782d3b9dd admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.476841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020f13e-1794-4967-888f-35b782d3b9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.477151 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020f13e-1794-4967-888f-35b782d3b9dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.478278 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020f13e-1794-4967-888f-35b782d3b9dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:45.480341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020f13e-1794-4967-888f-35b782d3b9dd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:45.501128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1045/2082] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:45.501827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020f13e-1794-4967-888f-35b782d3b9dd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:45.513890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020f13e-1794-4967-888f-35b782d3b9dd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:45.518959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7b461-8b2c-43d9-a3a4-d699f1e2f454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:45.519627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7b461-8b2c-43d9-a3a4-d699f1e2f454 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.520131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7b461-8b2c-43d9-a3a4-d699f1e2f454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.520295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1038/2083] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:29:45.520397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7b461-8b2c-43d9-a3a4-d699f1e2f454 admin admin] PATH_INFO: `/v3/projects/78948d9a0aff492ba4dc0657e1730467/users/ebdc6beeb54d426cb5956675146299e2/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.522808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7b461-8b2c-43d9-a3a4-d699f1e2f454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78948d9a0aff492ba4dc0657e1730467, user_id=ebdc6beeb54d426cb5956675146299e2, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:45.525743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7b461-8b2c-43d9-a3a4-d699f1e2f454 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:45.526076 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fc9cc-901c-4450-bb73-84c50d1073a5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:45.537570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fc9cc-901c-4450-bb73-84c50d1073a5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:45.549695 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1046/2084] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:45 2022] PUT /identity/v3/projects/78948d9a0aff492ba4dc0657e1730467/users/ebdc6beeb54d426cb5956675146299e2/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:45.551216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2ea1f-5ca1-4fa5-a22b-849d36cad703 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.551502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2ea1f-5ca1-4fa5-a22b-849d36cad703 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.551777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2ea1f-5ca1-4fa5-a22b-849d36cad703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.586876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b3f2ea1f-5ca1-4fa5-a22b-849d36cad703 None None] MFA Rules not processed for user `ebdc6beeb54d426cb5956675146299e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:45.588485 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b3f2ea1f-5ca1-4fa5-a22b-849d36cad703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:45.658469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73820d16-7e97-418f-bda8-883d164c6aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:45.661220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73820d16-7e97-418f-bda8-883d164c6aaf admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.661728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73820d16-7e97-418f-bda8-883d164c6aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.663419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73820d16-7e97-418f-bda8-883d164c6aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.664232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73820d16-7e97-418f-bda8-883d164c6aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:45.668795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73820d16-7e97-418f-bda8-883d164c6aaf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:45.689998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1047/2085] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:45.703749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2ea1f-5ca1-4fa5-a22b-849d36cad703 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:45.705479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b3f2ea1f-5ca1-4fa5-a22b-849d36cad703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:45.739529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1039/2086] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:45 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:45.744051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73820d16-7e97-418f-bda8-883d164c6aaf admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:45.815657 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2ea1f-5ca1-4fa5-a22b-849d36cad703 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:45.832273 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73820d16-7e97-418f-bda8-883d164c6aaf admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:45.838111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20af7a1-e4e1-4c45-a238-367a8978762b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:45.838813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20af7a1-e4e1-4c45-a238-367a8978762b admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.839165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20af7a1-e4e1-4c45-a238-367a8978762b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.839506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20af7a1-e4e1-4c45-a238-367a8978762b admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.840297 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20af7a1-e4e1-4c45-a238-367a8978762b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:45.842655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20af7a1-e4e1-4c45-a238-367a8978762b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:45.868044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:45.878758 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d6909-1f11-4e82-8037-5b9fe8feeb66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:45.880022 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d6909-1f11-4e82-8037-5b9fe8feeb66 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.880499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d6909-1f11-4e82-8037-5b9fe8feeb66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.880964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d6909-1f11-4e82-8037-5b9fe8feeb66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.884930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1040/2087] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:45 2022] POST /identity/v3/users => generated 299 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:29:45.888231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20af7a1-e4e1-4c45-a238-367a8978762b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:45.907144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20af7a1-e4e1-4c45-a238-367a8978762b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:45.909929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:45.922932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33a404-f342-4715-a1d3-b3bee87ca0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:45.923645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33a404-f342-4715-a1d3-b3bee87ca0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.924083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33a404-f342-4715-a1d3-b3bee87ca0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.924370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33a404-f342-4715-a1d3-b3bee87ca0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.925031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33a404-f342-4715-a1d3-b3bee87ca0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:45.928308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33a404-f342-4715-a1d3-b3bee87ca0b9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:45.948124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1041/2088] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:45.949628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33a404-f342-4715-a1d3-b3bee87ca0b9 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:45.962689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33a404-f342-4715-a1d3-b3bee87ca0b9 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:45.968336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cce63-4eb1-4cac-868d-961ff18a842b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:45.969018 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cce63-4eb1-4cac-868d-961ff18a842b admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:45.969361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cce63-4eb1-4cac-868d-961ff18a842b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:45.969637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cce63-4eb1-4cac-868d-961ff18a842b admin admin] PATH_INFO: `/v3/projects/f30f2b3afe88499f97b560aad4f6888a/users/ee167d73f8514588ac8ce7476cb0882f/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:45.986056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0cce63-4eb1-4cac-868d-961ff18a842b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f30f2b3afe88499f97b560aad4f6888a, user_id=ee167d73f8514588ac8ce7476cb0882f, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:45.989948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0cce63-4eb1-4cac-868d-961ff18a842b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:45.994332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d6909-1f11-4e82-8037-5b9fe8feeb66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:45.996303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d6909-1f11-4e82-8037-5b9fe8feeb66 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:46.010909 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1042/2089] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:45 2022] PUT /identity/v3/projects/f30f2b3afe88499f97b560aad4f6888a/users/ee167d73f8514588ac8ce7476cb0882f/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:46.013257 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1048/2090] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:29:46.014826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cce63-4eb1-4cac-868d-961ff18a842b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:46.124228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cce63-4eb1-4cac-868d-961ff18a842b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:46.143893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c1e2c-27d3-4121-b8b2-0db027718b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:46.144524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c1e2c-27d3-4121-b8b2-0db027718b3c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:46.144800 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c1e2c-27d3-4121-b8b2-0db027718b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:46.145064 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c1e2c-27d3-4121-b8b2-0db027718b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:46.145747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c1e2c-27d3-4121-b8b2-0db027718b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:46.147415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c1e2c-27d3-4121-b8b2-0db027718b3c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:46.167256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1043/2091] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:46.172309 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d6909-1f11-4e82-8037-5b9fe8feeb66 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:46.189708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d6909-1f11-4e82-8037-5b9fe8feeb66 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:46.198561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3bf89-4590-42e0-882d-837b1bedc2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:46.199495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3bf89-4590-42e0-882d-837b1bedc2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:46.199495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3bf89-4590-42e0-882d-837b1bedc2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:46.199696 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3bf89-4590-42e0-882d-837b1bedc2a7 admin admin] PATH_INFO: `/v3/projects/f30f2b3afe88499f97b560aad4f6888a/users/ee167d73f8514588ac8ce7476cb0882f/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:46.201803 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3bf89-4590-42e0-882d-837b1bedc2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f30f2b3afe88499f97b560aad4f6888a, user_id=ee167d73f8514588ac8ce7476cb0882f, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:46.204948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3bf89-4590-42e0-882d-837b1bedc2a7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:46.222246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1049/2092] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:46 2022] PUT /identity/v3/projects/f30f2b3afe88499f97b560aad4f6888a/users/ee167d73f8514588ac8ce7476cb0882f/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:46.236123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359c759-1e36-4749-92ed-1caf61b6868d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:46.236463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359c759-1e36-4749-92ed-1caf61b6868d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:46.239233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359c759-1e36-4749-92ed-1caf61b6868d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:46.331904 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5359c759-1e36-4749-92ed-1caf61b6868d None None] MFA Rules not processed for user `ee167d73f8514588ac8ce7476cb0882f`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:46.335423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5359c759-1e36-4749-92ed-1caf61b6868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:46.492302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1044/2093] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:46.502069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3bf89-4590-42e0-882d-837b1bedc2a7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:46.579136 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3bf89-4590-42e0-882d-837b1bedc2a7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:46.583740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94b50d-a216-4ff3-ab9a-d7f7024fe49e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:46.584656 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94b50d-a216-4ff3-ab9a-d7f7024fe49e service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:46.585029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94b50d-a216-4ff3-ab9a-d7f7024fe49e service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:46.585414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94b50d-a216-4ff3-ab9a-d7f7024fe49e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:46.660613 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94b50d-a216-4ff3-ab9a-d7f7024fe49e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:46.662265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94b50d-a216-4ff3-ab9a-d7f7024fe49e service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:46.676208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1050/2094] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:49.380482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1e447-49a8-47d7-ac32-3ee11a5f0947 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:49.381056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1e447-49a8-47d7-ac32-3ee11a5f0947 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:49.381269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1e447-49a8-47d7-ac32-3ee11a5f0947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:49.412783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-01d1e447-49a8-47d7-ac32-3ee11a5f0947 None None] MFA Rules not processed for user `8606d928811c4c64aea5db0c71a81c04`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:49.414153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-01d1e447-49a8-47d7-ac32-3ee11a5f0947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:49.456987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:49.500074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1045/2095] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:49.512560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94b50d-a216-4ff3-ab9a-d7f7024fe49e service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:49.514199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4a94b50d-a216-4ff3-ab9a-d7f7024fe49e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:49.585178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94b50d-a216-4ff3-ab9a-d7f7024fe49e service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:49.589594 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7f18b-d85a-4208-b6cb-3c1acd54f2fa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf8a0ae03c94aa78cc2ab0e4e1c87b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:49.590248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7f18b-d85a-4208-b6cb-3c1acd54f2fa service glance] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:49.590565 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7f18b-d85a-4208-b6cb-3c1acd54f2fa service glance] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:49.590909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7f18b-d85a-4208-b6cb-3c1acd54f2fa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:49.592883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ceb7f18b-d85a-4208-b6cb-3c1acd54f2fa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:49.599923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:49.699155 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7f18b-d85a-4208-b6cb-3c1acd54f2fa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:49.702944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7f18b-d85a-4208-b6cb-3c1acd54f2fa service glance] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:49.713787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1051/2096] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:49.981707 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085a764-f0c9-40ef-8d53-abb11a196d36 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:49.982181 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085a764-f0c9-40ef-8d53-abb11a196d36 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:49.982547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085a764-f0c9-40ef-8d53-abb11a196d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:50.006836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6085a764-f0c9-40ef-8d53-abb11a196d36 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:50.008169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6085a764-f0c9-40ef-8d53-abb11a196d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:50.066279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1046/2097] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:29:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:50.106628 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7f18b-d85a-4208-b6cb-3c1acd54f2fa service glance] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:50.108164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ceb7f18b-d85a-4208-b6cb-3c1acd54f2fa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:50.180616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7f18b-d85a-4208-b6cb-3c1acd54f2fa service glance] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:50.185561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b23247-abf2-4bcf-af72-ed57db35b7d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:50.186163 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b23247-abf2-4bcf-af72-ed57db35b7d6 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:50.186434 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b23247-abf2-4bcf-af72-ed57db35b7d6 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:50.186694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b23247-abf2-4bcf-af72-ed57db35b7d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:50.194568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b23247-abf2-4bcf-af72-ed57db35b7d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:50.196166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b23247-abf2-4bcf-af72-ed57db35b7d6 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:50.203901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1052/2098] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:29:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:29:56.320727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085a764-f0c9-40ef-8d53-abb11a196d36 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:56.324124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6085a764-f0c9-40ef-8d53-abb11a196d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:56.375332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085a764-f0c9-40ef-8d53-abb11a196d36 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:56.381739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a67e3-f0db-4d60-b92a-034bed050d45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:56.383649 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a67e3-f0db-4d60-b92a-034bed050d45 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:56.383935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a67e3-f0db-4d60-b92a-034bed050d45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:56.384211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a67e3-f0db-4d60-b92a-034bed050d45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:56.508659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a67e3-f0db-4d60-b92a-034bed050d45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:56.512224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a67e3-f0db-4d60-b92a-034bed050d45 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:56.523579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1047/2099] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:56.659496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b23247-abf2-4bcf-af72-ed57db35b7d6 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:56.778780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b23247-abf2-4bcf-af72-ed57db35b7d6 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:56.784519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:56.785389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:56.785675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:56.786108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] PATH_INFO: `/v3/users/893046830f204e00966b35a8e223c9ce` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:56.787341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=893046830f204e00966b35a8e223c9ce)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:56.794359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:56.863993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:56.874358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893046830f204e00966b35a8e223c9ce'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:56.886516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893046830f204e00966b35a8e223c9ce'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:56.888175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893046830f204e00966b35a8e223c9ce'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:56.900721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893046830f204e00966b35a8e223c9ce'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:56.921650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893046830f204e00966b35a8e223c9ce'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:56.937381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1053/2100] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:56 2022] DELETE /identity/v3/users/893046830f204e00966b35a8e223c9ce => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:56.940824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0d1c0-aa6e-4d16-b45a-43ab13d58e6d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:56.941348 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0d1c0-aa6e-4d16-b45a-43ab13d58e6d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:56.941614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0d1c0-aa6e-4d16-b45a-43ab13d58e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:56.964552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9af0d1c0-aa6e-4d16-b45a-43ab13d58e6d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:56.965933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9af0d1c0-aa6e-4d16-b45a-43ab13d58e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:57.058985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1048/2101] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:29:57.067901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:57.069456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:57.197789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7453dc3-d28c-43ef-a1e1-77190314a482 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:57.201774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9be9bb-485f-44f9-bd3a-6781321f81b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:57.202400 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9be9bb-485f-44f9-bd3a-6781321f81b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:57.202680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9be9bb-485f-44f9-bd3a-6781321f81b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:57.202954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9be9bb-485f-44f9-bd3a-6781321f81b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:57.214961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9be9bb-485f-44f9-bd3a-6781321f81b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:57.216637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9be9bb-485f-44f9-bd3a-6781321f81b1 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:57.228761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1054/2102] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:57.477360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0d1c0-aa6e-4d16-b45a-43ab13d58e6d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:57.482511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9af0d1c0-aa6e-4d16-b45a-43ab13d58e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:57.506333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:57.616290 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0d1c0-aa6e-4d16-b45a-43ab13d58e6d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:57.628230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c3b9a-1a03-4fc9-9077-92324f225dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:57.629007 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c3b9a-1a03-4fc9-9077-92324f225dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:57.629007 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c3b9a-1a03-4fc9-9077-92324f225dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:57.629319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c3b9a-1a03-4fc9-9077-92324f225dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:57.633997 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-117c3b9a-1a03-4fc9-9077-92324f225dc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:57.802145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c3b9a-1a03-4fc9-9077-92324f225dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:57.804012 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9be9bb-485f-44f9-bd3a-6781321f81b1 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:57.805884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fd9be9bb-485f-44f9-bd3a-6781321f81b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:57.816705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c3b9a-1a03-4fc9-9077-92324f225dc3 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:57.836773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1049/2103] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:29:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:29:57.845931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9be9bb-485f-44f9-bd3a-6781321f81b1 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:57.851430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:57.852065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:57.852332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:57.852603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] PATH_INFO: `/v3/projects/0782cfa4b62c4dc1a1fc9c241e0cb606` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:57.853881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0782cfa4b62c4dc1a1fc9c241e0cb606)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:57.855852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:57.929429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0782cfa4b62c4dc1a1fc9c241e0cb606', 'request_id': 'req-0add4454-ba89-4e91-8d7f-1005118bb6fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:57.940540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0782cfa4b62c4dc1a1fc9c241e0cb606', 'request_id': 'req-0add4454-ba89-4e91-8d7f-1005118bb6fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:57.950668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0782cfa4b62c4dc1a1fc9c241e0cb606', 'request_id': 'req-0add4454-ba89-4e91-8d7f-1005118bb6fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:57.962693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] The token cache is being invalidate because project 0782cfa4b62c4dc1a1fc9c241e0cb606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:57.963094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:57.964494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:57.969620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1055/2104] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:57 2022] DELETE /identity/v3/projects/0782cfa4b62c4dc1a1fc9c241e0cb606 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:57.970895 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c3b9a-1a03-4fc9-9077-92324f225dc3 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:57.984159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c3b9a-1a03-4fc9-9077-92324f225dc3 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:58.063471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16af50-fd5f-4ae8-b7a2-83af33578f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:58.064557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16af50-fd5f-4ae8-b7a2-83af33578f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:58.065037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16af50-fd5f-4ae8-b7a2-83af33578f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:58.065576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16af50-fd5f-4ae8-b7a2-83af33578f76 admin admin] PATH_INFO: `/v3/users/ebdc6beeb54d426cb5956675146299e2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:58.066924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16af50-fd5f-4ae8-b7a2-83af33578f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebdc6beeb54d426cb5956675146299e2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:58.069193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16af50-fd5f-4ae8-b7a2-83af33578f76 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:58.135115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8e16af50-fd5f-4ae8-b7a2-83af33578f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebdc6beeb54d426cb5956675146299e2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:58.143773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8e16af50-fd5f-4ae8-b7a2-83af33578f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebdc6beeb54d426cb5956675146299e2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:58.144529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8e16af50-fd5f-4ae8-b7a2-83af33578f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebdc6beeb54d426cb5956675146299e2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:58.155005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8e16af50-fd5f-4ae8-b7a2-83af33578f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebdc6beeb54d426cb5956675146299e2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:58.157331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:58.172877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8e16af50-fd5f-4ae8-b7a2-83af33578f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebdc6beeb54d426cb5956675146299e2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:58.179742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1050/2105] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:29:57 2022] DELETE /identity/v3/users/ebdc6beeb54d426cb5956675146299e2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:58.520951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:58.523179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:58.548507 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add4454-ba89-4e91-8d7f-1005118bb6fd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:58.597665 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:58.624494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:58.625787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:58.626004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:58.626432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b admin admin] PATH_INFO: `/v3/projects/78948d9a0aff492ba4dc0657e1730467` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:58.628956 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=78948d9a0aff492ba4dc0657e1730467)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:58.631955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:58.719114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78948d9a0aff492ba4dc0657e1730467', 'request_id': 'req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:58.728785 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78948d9a0aff492ba4dc0657e1730467', 'request_id': 'req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:58.736515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78948d9a0aff492ba4dc0657e1730467', 'request_id': 'req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:58.748967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b admin admin] The token cache is being invalidate because project 78948d9a0aff492ba4dc0657e1730467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:29:58.749796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:58.756608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9df90479-59f7-4a0b-bef8-3d251bfe0b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:29:58.759175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1056/2106] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:29:58 2022] DELETE /identity/v3/projects/78948d9a0aff492ba4dc0657e1730467 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:58.764632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d449e-a377-4363-9cb8-0a2a1e4c593e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:58.765166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d449e-a377-4363-9cb8-0a2a1e4c593e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:58.765249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d449e-a377-4363-9cb8-0a2a1e4c593e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:58.788354 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-8e1d449e-a377-4363-9cb8-0a2a1e4c593e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:58.789678 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8e1d449e-a377-4363-9cb8-0a2a1e4c593e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:58.903448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1051/2107] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:29:58.911977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aece00-a6ce-406f-af2f-36d492a363c8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:58.912478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aece00-a6ce-406f-af2f-36d492a363c8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:58.912478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aece00-a6ce-406f-af2f-36d492a363c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:58.936704 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a5aece00-a6ce-406f-af2f-36d492a363c8 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:58.938060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a5aece00-a6ce-406f-af2f-36d492a363c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:58.951430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d449e-a377-4363-9cb8-0a2a1e4c593e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:58.958886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1057/2108] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:58.963515 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d449e-a377-4363-9cb8-0a2a1e4c593e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:58.970514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8255a90-5e98-40ec-8a89-3daf99d360da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:58.971190 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8255a90-5e98-40ec-8a89-3daf99d360da admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:58.971471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8255a90-5e98-40ec-8a89-3daf99d360da admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:58.971739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8255a90-5e98-40ec-8a89-3daf99d360da admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:58.973810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8255a90-5e98-40ec-8a89-3daf99d360da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:29:58.975159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8255a90-5e98-40ec-8a89-3daf99d360da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:58.991560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8255a90-5e98-40ec-8a89-3daf99d360da admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:59.034979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aece00-a6ce-406f-af2f-36d492a363c8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:59.041128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1052/2109] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:29:59.050288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aece00-a6ce-406f-af2f-36d492a363c8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:59.055360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b6d54-9263-41c7-96af-ca364c39ff51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:59.055965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b6d54-9263-41c7-96af-ca364c39ff51 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:59.056245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b6d54-9263-41c7-96af-ca364c39ff51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:59.056511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b6d54-9263-41c7-96af-ca364c39ff51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:59.061263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b6d54-9263-41c7-96af-ca364c39ff51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:59.061263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b6d54-9263-41c7-96af-ca364c39ff51 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:59.111823 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1058/2110] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:59 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:59.116007 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8255a90-5e98-40ec-8a89-3daf99d360da admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:59.134305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8255a90-5e98-40ec-8a89-3daf99d360da admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:59.282428 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601958b4-4203-40ba-acbc-87c3289d4d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:59.283900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601958b4-4203-40ba-acbc-87c3289d4d1d admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:59.284359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601958b4-4203-40ba-acbc-87c3289d4d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:59.284755 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601958b4-4203-40ba-acbc-87c3289d4d1d admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:59.288765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601958b4-4203-40ba-acbc-87c3289d4d1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:59.298851 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601958b4-4203-40ba-acbc-87c3289d4d1d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:59.340177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1053/2111] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:59 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:59.344318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b6d54-9263-41c7-96af-ca364c39ff51 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:59.360896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b6d54-9263-41c7-96af-ca364c39ff51 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:59.370160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cce82a-c7cb-4443-a75b-8e5f09a6db3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:59.371087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cce82a-c7cb-4443-a75b-8e5f09a6db3d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:59.371627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cce82a-c7cb-4443-a75b-8e5f09a6db3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:59.371866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cce82a-c7cb-4443-a75b-8e5f09a6db3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:59.372592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cce82a-c7cb-4443-a75b-8e5f09a6db3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:59.374377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cce82a-c7cb-4443-a75b-8e5f09a6db3d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:59.396886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1059/2112] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:29:59.397274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601958b4-4203-40ba-acbc-87c3289d4d1d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:59.413458 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601958b4-4203-40ba-acbc-87c3289d4d1d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:59.418303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de0e63-8094-4aac-8d6d-287cd23f4e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:59.418586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de0e63-8094-4aac-8d6d-287cd23f4e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:59.418932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de0e63-8094-4aac-8d6d-287cd23f4e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:59.419191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de0e63-8094-4aac-8d6d-287cd23f4e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:59.419997 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de0e63-8094-4aac-8d6d-287cd23f4e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:59.421903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de0e63-8094-4aac-8d6d-287cd23f4e29 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:59.439253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cce82a-c7cb-4443-a75b-8e5f09a6db3d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:59.445998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1054/2113] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:29:59.451604 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cce82a-c7cb-4443-a75b-8e5f09a6db3d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:59.456836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f79269-70a9-4184-bb76-f1ccecca17a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:59.457637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f79269-70a9-4184-bb76-f1ccecca17a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:59.457917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f79269-70a9-4184-bb76-f1ccecca17a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:59.458188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f79269-70a9-4184-bb76-f1ccecca17a6 admin admin] PATH_INFO: `/v3/projects/f8bb6374238e40688486fe4a26879f65/users/6a1834aabab24c2694ab7f7dbab936a4/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:59.485472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f79269-70a9-4184-bb76-f1ccecca17a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8bb6374238e40688486fe4a26879f65, user_id=6a1834aabab24c2694ab7f7dbab936a4, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:59.492064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f79269-70a9-4184-bb76-f1ccecca17a6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:59.551230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d02e15-959f-46d8-8ba6-8bb960f95839 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:59.551623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d02e15-959f-46d8-8ba6-8bb960f95839 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:59.552054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d02e15-959f-46d8-8ba6-8bb960f95839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:59.562356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1060/2114] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:29:59 2022] PUT /identity/v3/projects/f8bb6374238e40688486fe4a26879f65/users/6a1834aabab24c2694ab7f7dbab936a4/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:29:59.606149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-60d02e15-959f-46d8-8ba6-8bb960f95839 None None] MFA Rules not processed for user `6a1834aabab24c2694ab7f7dbab936a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:29:59.607404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-60d02e15-959f-46d8-8ba6-8bb960f95839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:29:59.657416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:59.694900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1055/2115] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:29:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:29:59.699624 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f79269-70a9-4184-bb76-f1ccecca17a6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:59.777885 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f79269-70a9-4184-bb76-f1ccecca17a6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:59.782757 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a260ca5-ee29-4eb4-b78c-a15deb2c7518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:59.783704 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a260ca5-ee29-4eb4-b78c-a15deb2c7518 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:59.784213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a260ca5-ee29-4eb4-b78c-a15deb2c7518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:59.784920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a260ca5-ee29-4eb4-b78c-a15deb2c7518 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:59.785752 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a260ca5-ee29-4eb4-b78c-a15deb2c7518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:59.790447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a260ca5-ee29-4eb4-b78c-a15deb2c7518 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:59.805934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:59.827326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1061/2116] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:29:59 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:59.832029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d02e15-959f-46d8-8ba6-8bb960f95839 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:59.845826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d02e15-959f-46d8-8ba6-8bb960f95839 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:59.912711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569aab2-29d4-4485-b1be-0e027509261f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:59.913462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569aab2-29d4-4485-b1be-0e027509261f admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:59.914147 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569aab2-29d4-4485-b1be-0e027509261f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:59.914541 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569aab2-29d4-4485-b1be-0e027509261f admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:59.915419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569aab2-29d4-4485-b1be-0e027509261f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:59.917788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569aab2-29d4-4485-b1be-0e027509261f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:59.945842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1056/2117] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:29:59 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:29:59.948585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a260ca5-ee29-4eb4-b78c-a15deb2c7518 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:29:59.965201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a260ca5-ee29-4eb4-b78c-a15deb2c7518 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:29:59.971882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cc465-ec7c-4616-81b9-ed49d42e5044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:29:59.972823 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cc465-ec7c-4616-81b9-ed49d42e5044 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:29:59.973243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cc465-ec7c-4616-81b9-ed49d42e5044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:29:59.973649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cc465-ec7c-4616-81b9-ed49d42e5044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:29:59.974495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cc465-ec7c-4616-81b9-ed49d42e5044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:29:59.976554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cc465-ec7c-4616-81b9-ed49d42e5044 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:29:59.994460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1062/2118] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:29:59.995773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569aab2-29d4-4485-b1be-0e027509261f admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:00.006825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569aab2-29d4-4485-b1be-0e027509261f admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:00.011903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfebd8-1e4a-455e-b6bb-90956dd08b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:00.012747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfebd8-1e4a-455e-b6bb-90956dd08b8e admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:00.013041 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfebd8-1e4a-455e-b6bb-90956dd08b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:00.013406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfebd8-1e4a-455e-b6bb-90956dd08b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:00.014199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfebd8-1e4a-455e-b6bb-90956dd08b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:00.016074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfebd8-1e4a-455e-b6bb-90956dd08b8e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:00.030592 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1057/2119] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:30:00.031620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cc465-ec7c-4616-81b9-ed49d42e5044 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:00.042876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cc465-ec7c-4616-81b9-ed49d42e5044 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:00.049359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f9884-5d60-4ab1-8df1-a2c4b913ef3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:00.050099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f9884-5d60-4ab1-8df1-a2c4b913ef3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:00.050370 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f9884-5d60-4ab1-8df1-a2c4b913ef3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:00.050639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f9884-5d60-4ab1-8df1-a2c4b913ef3a admin admin] PATH_INFO: `/v3/projects/872071de909b45c096dc64279bdf7205/users/d9d404df955b4d8283139f3ad12b9880/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:00.076418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f9884-5d60-4ab1-8df1-a2c4b913ef3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=872071de909b45c096dc64279bdf7205, user_id=d9d404df955b4d8283139f3ad12b9880, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:00.079716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f9884-5d60-4ab1-8df1-a2c4b913ef3a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:00.095447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1063/2120] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:30:00 2022] PUT /identity/v3/projects/872071de909b45c096dc64279bdf7205/users/d9d404df955b4d8283139f3ad12b9880/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:00.106113 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc9664-08ef-400f-bfd4-b37cc14b9a37 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:00.106399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc9664-08ef-400f-bfd4-b37cc14b9a37 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:00.106710 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc9664-08ef-400f-bfd4-b37cc14b9a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:00.149866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2efc9664-08ef-400f-bfd4-b37cc14b9a37 None None] MFA Rules not processed for user `d9d404df955b4d8283139f3ad12b9880`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:00.151834 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2efc9664-08ef-400f-bfd4-b37cc14b9a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:00.237805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1058/2121] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:00.242048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f9884-5d60-4ab1-8df1-a2c4b913ef3a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:00.316448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f9884-5d60-4ab1-8df1-a2c4b913ef3a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:00.323344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a1321-bfca-45e7-8539-53134b10950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:00.324286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a1321-bfca-45e7-8539-53134b10950b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:00.324782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a1321-bfca-45e7-8539-53134b10950b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:00.325287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a1321-bfca-45e7-8539-53134b10950b admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:00.326408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a1321-bfca-45e7-8539-53134b10950b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:00.328742 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a1321-bfca-45e7-8539-53134b10950b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:00.383226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1064/2122] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:30:00 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:30:00.393957 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc9664-08ef-400f-bfd4-b37cc14b9a37 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:00.409875 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc9664-08ef-400f-bfd4-b37cc14b9a37 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:00.496994 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:00.594202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb90e07-720b-4cb6-8caa-9e6e3e27bc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:00.594839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb90e07-720b-4cb6-8caa-9e6e3e27bc99 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:00.595158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb90e07-720b-4cb6-8caa-9e6e3e27bc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:00.595425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb90e07-720b-4cb6-8caa-9e6e3e27bc99 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:00.596090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb90e07-720b-4cb6-8caa-9e6e3e27bc99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:00.598220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb90e07-720b-4cb6-8caa-9e6e3e27bc99 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:00.642859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1059/2123] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:30:00 2022] POST /identity/v3/users => generated 298 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:30:00.644497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a1321-bfca-45e7-8539-53134b10950b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:00.659150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a1321-bfca-45e7-8539-53134b10950b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:00.666987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b24cc5-e70a-4227-baed-c605ad10d811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:00.668069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b24cc5-e70a-4227-baed-c605ad10d811 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:00.668486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b24cc5-e70a-4227-baed-c605ad10d811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:00.668909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b24cc5-e70a-4227-baed-c605ad10d811 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:00.669788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b24cc5-e70a-4227-baed-c605ad10d811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:00.672525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b24cc5-e70a-4227-baed-c605ad10d811 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:00.689788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1065/2124] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:30:00.693526 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb90e07-720b-4cb6-8caa-9e6e3e27bc99 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:00.709148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb90e07-720b-4cb6-8caa-9e6e3e27bc99 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:00.714463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f518af-9055-42d5-b8ca-5590b551ff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:00.715133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f518af-9055-42d5-b8ca-5590b551ff88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:00.715462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f518af-9055-42d5-b8ca-5590b551ff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:00.715729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f518af-9055-42d5-b8ca-5590b551ff88 admin admin] PATH_INFO: `/v3/projects/d33c55d1763e457392c5b504e5aff70f/users/cb9fe17b7a8c4c13bc7f2f1cc8b663c8/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:00.740546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f518af-9055-42d5-b8ca-5590b551ff88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33c55d1763e457392c5b504e5aff70f, user_id=cb9fe17b7a8c4c13bc7f2f1cc8b663c8, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:00.745244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f518af-9055-42d5-b8ca-5590b551ff88 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:00.775222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1060/2125] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:30:00 2022] PUT /identity/v3/projects/d33c55d1763e457392c5b504e5aff70f/users/cb9fe17b7a8c4c13bc7f2f1cc8b663c8/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:00.779306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b24cc5-e70a-4227-baed-c605ad10d811 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:00.795043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b24cc5-e70a-4227-baed-c605ad10d811 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:00.916901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21133d-da03-4396-a1e2-3b70cd9226ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:00.917614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21133d-da03-4396-a1e2-3b70cd9226ef admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:00.917897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21133d-da03-4396-a1e2-3b70cd9226ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:00.918328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21133d-da03-4396-a1e2-3b70cd9226ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:00.919031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21133d-da03-4396-a1e2-3b70cd9226ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:00.921530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21133d-da03-4396-a1e2-3b70cd9226ef admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:00.937938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1066/2126] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:30:00.938598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f518af-9055-42d5-b8ca-5590b551ff88 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:00.954542 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f518af-9055-42d5-b8ca-5590b551ff88 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:00.971517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6ecfe-1b4d-4560-88c3-1c210e31a854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:00.972383 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6ecfe-1b4d-4560-88c3-1c210e31a854 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:00.972662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6ecfe-1b4d-4560-88c3-1c210e31a854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:00.972933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6ecfe-1b4d-4560-88c3-1c210e31a854 admin admin] PATH_INFO: `/v3/projects/d33c55d1763e457392c5b504e5aff70f/users/cb9fe17b7a8c4c13bc7f2f1cc8b663c8/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:00.977841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6ecfe-1b4d-4560-88c3-1c210e31a854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33c55d1763e457392c5b504e5aff70f, user_id=cb9fe17b7a8c4c13bc7f2f1cc8b663c8, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:00.980710 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6ecfe-1b4d-4560-88c3-1c210e31a854 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:01.003689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1061/2127] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:30:00 2022] PUT /identity/v3/projects/d33c55d1763e457392c5b504e5aff70f/users/cb9fe17b7a8c4c13bc7f2f1cc8b663c8/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:01.015973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363136dd-ed96-4e1d-8166-f3dec78af029 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:01.016576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363136dd-ed96-4e1d-8166-f3dec78af029 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:01.016576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363136dd-ed96-4e1d-8166-f3dec78af029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:01.073520 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-363136dd-ed96-4e1d-8166-f3dec78af029 None None] MFA Rules not processed for user `cb9fe17b7a8c4c13bc7f2f1cc8b663c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:01.074928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-363136dd-ed96-4e1d-8166-f3dec78af029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:01.105778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:01.225960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1067/2128] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:01 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:01.284645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6ecfe-1b4d-4560-88c3-1c210e31a854 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:01.286705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-dfe6ecfe-1b4d-4560-88c3-1c210e31a854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:01.394976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6ecfe-1b4d-4560-88c3-1c210e31a854 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:01.403615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17f551-9d04-455f-9acb-99559d9b0d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:01.404436 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17f551-9d04-455f-9acb-99559d9b0d3f service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:01.404879 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17f551-9d04-455f-9acb-99559d9b0d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:01.405342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17f551-9d04-455f-9acb-99559d9b0d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:01.487538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17f551-9d04-455f-9acb-99559d9b0d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:01.489230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17f551-9d04-455f-9acb-99559d9b0d3f service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:01.498981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1062/2129] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:30:01.795582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26795d-70c7-4196-aaf4-c3129d4120d6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:01.796349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26795d-70c7-4196-aaf4-c3129d4120d6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:01.796349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26795d-70c7-4196-aaf4-c3129d4120d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:01.821533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4e26795d-70c7-4196-aaf4-c3129d4120d6 None None] MFA Rules not processed for user `f6a5aff543234d7a9976c2e43453aef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:01.822785 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4e26795d-70c7-4196-aaf4-c3129d4120d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:01.912985 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1068/2130] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:02.553775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17f551-9d04-455f-9acb-99559d9b0d3f service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:02.555376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5f17f551-9d04-455f-9acb-99559d9b0d3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:02.692327 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:02.724224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17f551-9d04-455f-9acb-99559d9b0d3f service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:02.738235 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:02.739013 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:02.739371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:02.740122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] PATH_INFO: `/v3/users/d655cf758307466a9d58b1f16a7a6cbb` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:02.744352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d655cf758307466a9d58b1f16a7a6cbb)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:02.748617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:02.857218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd655cf758307466a9d58b1f16a7a6cbb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:02.868639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd655cf758307466a9d58b1f16a7a6cbb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:02.869883 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd655cf758307466a9d58b1f16a7a6cbb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:02.880234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd655cf758307466a9d58b1f16a7a6cbb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:02.897925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd655cf758307466a9d58b1f16a7a6cbb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:02.905363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1063/2131] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:30:02 2022] DELETE /identity/v3/users/d655cf758307466a9d58b1f16a7a6cbb => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:02.911390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e368736-9271-4297-ad27-5ec46025eeb7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:02.911913 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e368736-9271-4297-ad27-5ec46025eeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:02.912164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e368736-9271-4297-ad27-5ec46025eeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:02.942526 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7e368736-9271-4297-ad27-5ec46025eeb7 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:02.944425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7e368736-9271-4297-ad27-5ec46025eeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:03.122461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1069/2132] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:03.129025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:03.130574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:03.218999 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb224e-3786-4b83-ba6e-9b6891149a61 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:03.222088 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80dbcea-bce1-4057-aeb0-54dd516718b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:03.223143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80dbcea-bce1-4057-aeb0-54dd516718b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:03.223794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80dbcea-bce1-4057-aeb0-54dd516718b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:03.224156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80dbcea-bce1-4057-aeb0-54dd516718b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:03.239125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80dbcea-bce1-4057-aeb0-54dd516718b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:03.240949 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80dbcea-bce1-4057-aeb0-54dd516718b6 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:03.256436 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1064/2133] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:30:03.259266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e368736-9271-4297-ad27-5ec46025eeb7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:03.260818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7e368736-9271-4297-ad27-5ec46025eeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:03.379559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e368736-9271-4297-ad27-5ec46025eeb7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:03.392539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed3054-e3b3-4f2b-b9cc-0ca12bde06fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:03.393401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed3054-e3b3-4f2b-b9cc-0ca12bde06fa service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:03.393856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed3054-e3b3-4f2b-b9cc-0ca12bde06fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:03.394269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed3054-e3b3-4f2b-b9cc-0ca12bde06fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:03.396519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-08ed3054-e3b3-4f2b-b9cc-0ca12bde06fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:03.412636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:03.497938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed3054-e3b3-4f2b-b9cc-0ca12bde06fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:03.500218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed3054-e3b3-4f2b-b9cc-0ca12bde06fa service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:03.530474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1070/2134] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:30:03.684605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80dbcea-bce1-4057-aeb0-54dd516718b6 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:03.686255 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d80dbcea-bce1-4057-aeb0-54dd516718b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:03.714262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80dbcea-bce1-4057-aeb0-54dd516718b6 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:03.725961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:03.727086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:03.727086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:03.727420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] PATH_INFO: `/v3/projects/d4c2523a071e49938bea216b99d68bac` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:03.729054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4c2523a071e49938bea216b99d68bac)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:03.732481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:03.840153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4c2523a071e49938bea216b99d68bac', 'request_id': 'req-9b0cf160-602c-4c93-87fc-bacbec3d9f53'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:03.875081 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4c2523a071e49938bea216b99d68bac', 'request_id': 'req-9b0cf160-602c-4c93-87fc-bacbec3d9f53'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:03.888016 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4c2523a071e49938bea216b99d68bac', 'request_id': 'req-9b0cf160-602c-4c93-87fc-bacbec3d9f53'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:03.904738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:03.909720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] The token cache is being invalidate because project d4c2523a071e49938bea216b99d68bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:30:03.910011 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:03.910691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:03.913149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1065/2135] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:30:03 2022] DELETE /identity/v3/projects/d4c2523a071e49938bea216b99d68bac => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:03.917262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed3054-e3b3-4f2b-b9cc-0ca12bde06fa service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:03.948453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed3054-e3b3-4f2b-b9cc-0ca12bde06fa service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:04.042150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:04.042803 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:04.043081 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:04.043356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] PATH_INFO: `/v3/users/e17323fdffde4e75a1e1fa7757822528` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:04.044495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e17323fdffde4e75a1e1fa7757822528)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:04.046465 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:04.098072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e17323fdffde4e75a1e1fa7757822528'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.106742 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e17323fdffde4e75a1e1fa7757822528'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.107356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e17323fdffde4e75a1e1fa7757822528'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.114888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e17323fdffde4e75a1e1fa7757822528'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.127268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e17323fdffde4e75a1e1fa7757822528'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.133444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1071/2136] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:30:03 2022] DELETE /identity/v3/users/e17323fdffde4e75a1e1fa7757822528 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:04.434044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:04.435616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:04.473735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cf160-602c-4c93-87fc-bacbec3d9f53 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:04.554172 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:04.555197 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:04.555657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:04.556090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] PATH_INFO: `/v3/projects/a553772c3e474fcd80de0d3a3b514dfd` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:04.557638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a553772c3e474fcd80de0d3a3b514dfd)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:04.559861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:04.641429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a553772c3e474fcd80de0d3a3b514dfd', 'request_id': 'req-10c23a7b-2107-43fb-bbb6-091275e1db39'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.654149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a553772c3e474fcd80de0d3a3b514dfd', 'request_id': 'req-10c23a7b-2107-43fb-bbb6-091275e1db39'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.676401 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a553772c3e474fcd80de0d3a3b514dfd', 'request_id': 'req-10c23a7b-2107-43fb-bbb6-091275e1db39'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.693874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] The token cache is being invalidate because project a553772c3e474fcd80de0d3a3b514dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:30:04.694815 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.695591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.700293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1066/2137] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:30:04 2022] DELETE /identity/v3/projects/a553772c3e474fcd80de0d3a3b514dfd => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:04.701547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:04.703132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:04.717728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb45ff-26ab-4315-bf27-e46537664662 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:04.765647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:04.806434 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:04.807216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:04.807358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:04.807971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] PATH_INFO: `/v3/users/ee167d73f8514588ac8ce7476cb0882f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:04.808819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee167d73f8514588ac8ce7476cb0882f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:04.810953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:04.884089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee167d73f8514588ac8ce7476cb0882f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.904286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee167d73f8514588ac8ce7476cb0882f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.904987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee167d73f8514588ac8ce7476cb0882f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.918763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee167d73f8514588ac8ce7476cb0882f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.936412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee167d73f8514588ac8ce7476cb0882f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:04.946099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1072/2138] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:30:04 2022] DELETE /identity/v3/users/ee167d73f8514588ac8ce7476cb0882f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:05.131880 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:05.133576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:05.157160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c23a7b-2107-43fb-bbb6-091275e1db39 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:05.226634 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5b9ca-df35-4228-84ed-9c32197d8e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:05.227356 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5b9ca-df35-4228-84ed-9c32197d8e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.227572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5b9ca-df35-4228-84ed-9c32197d8e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.227840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5b9ca-df35-4228-84ed-9c32197d8e3b admin admin] PATH_INFO: `/v3/users/9be671f062ed4cb793facf9ca7920263` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.228991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5b9ca-df35-4228-84ed-9c32197d8e3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9be671f062ed4cb793facf9ca7920263)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:05.230980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5b9ca-df35-4228-84ed-9c32197d8e3b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:05.266647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:05.284616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aef5b9ca-df35-4228-84ed-9c32197d8e3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9be671f062ed4cb793facf9ca7920263'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.295532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aef5b9ca-df35-4228-84ed-9c32197d8e3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9be671f062ed4cb793facf9ca7920263'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.296381 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aef5b9ca-df35-4228-84ed-9c32197d8e3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9be671f062ed4cb793facf9ca7920263'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.308531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aef5b9ca-df35-4228-84ed-9c32197d8e3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9be671f062ed4cb793facf9ca7920263'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.317513 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:05.319216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:05.331883 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aef5b9ca-df35-4228-84ed-9c32197d8e3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9be671f062ed4cb793facf9ca7920263'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.336581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e8b85-0b1c-457e-883c-d4fe2c353094 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:05.340985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1067/2139] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:30:05 2022] DELETE /identity/v3/users/9be671f062ed4cb793facf9ca7920263 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:05.347528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:05.348374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.348529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.348995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] PATH_INFO: `/v3/projects/f30f2b3afe88499f97b560aad4f6888a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.350433 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f30f2b3afe88499f97b560aad4f6888a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:05.353904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:05.410682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f30f2b3afe88499f97b560aad4f6888a', 'request_id': 'req-8346851c-ec63-48c6-af8f-354fbcd57ff7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.421794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f30f2b3afe88499f97b560aad4f6888a', 'request_id': 'req-8346851c-ec63-48c6-af8f-354fbcd57ff7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.430860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f30f2b3afe88499f97b560aad4f6888a', 'request_id': 'req-8346851c-ec63-48c6-af8f-354fbcd57ff7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.442591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] The token cache is being invalidate because project f30f2b3afe88499f97b560aad4f6888a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:30:05.442880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.443529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.445120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1073/2140] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:30:05 2022] DELETE /identity/v3/projects/f30f2b3afe88499f97b560aad4f6888a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:05.455003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ee499-ce63-4604-bca8-e0f469313a8a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.455421 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ee499-ce63-4604-bca8-e0f469313a8a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.455790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ee499-ce63-4604-bca8-e0f469313a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.479568 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d12ee499-ce63-4604-bca8-e0f469313a8a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:05.480758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d12ee499-ce63-4604-bca8-e0f469313a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:05.529716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:05.531475 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:05.552926 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1068/2141] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:05.555063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ab79a-5336-491f-a94a-b8a9bec740a2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.555486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ab79a-5336-491f-a94a-b8a9bec740a2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.555659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ab79a-5336-491f-a94a-b8a9bec740a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.576513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9b8ab79a-5336-491f-a94a-b8a9bec740a2 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:05.577870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9b8ab79a-5336-491f-a94a-b8a9bec740a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:05.578345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:05.589123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1069/2142] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:05.591599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ab79a-5336-491f-a94a-b8a9bec740a2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:05.600991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ab79a-5336-491f-a94a-b8a9bec740a2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:05.605552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c140e4-5222-437f-877a-2591c10a3219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:05.606140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c140e4-5222-437f-877a-2591c10a3219 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.606415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c140e4-5222-437f-877a-2591c10a3219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.606684 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c140e4-5222-437f-877a-2591c10a3219 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.607332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c140e4-5222-437f-877a-2591c10a3219 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:30:05.607635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c140e4-5222-437f-877a-2591c10a3219 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:05.609256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c140e4-5222-437f-877a-2591c10a3219 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:05.615103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346851c-ec63-48c6-af8f-354fbcd57ff7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:05.620540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aba2f-64cd-403d-a8b8-69c42c134bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:05.621212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aba2f-64cd-403d-a8b8-69c42c134bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.621494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aba2f-64cd-403d-a8b8-69c42c134bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.621799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1070/2143] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:30:05.621912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aba2f-64cd-403d-a8b8-69c42c134bed admin admin] PATH_INFO: `/v3/projects/39cc778e84014ed98f33554b7c46b174` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.623119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59aba2f-64cd-403d-a8b8-69c42c134bed admin admin] RBAC: Authorizing `identity:delete_project(project_id=39cc778e84014ed98f33554b7c46b174)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:05.625064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59aba2f-64cd-403d-a8b8-69c42c134bed admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:05.625744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c140e4-5222-437f-877a-2591c10a3219 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:05.635741 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c140e4-5222-437f-877a-2591c10a3219 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:05.640592 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b9bac-90e5-422f-9f3a-02e2f73f6cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:05.641288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b9bac-90e5-422f-9f3a-02e2f73f6cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.641562 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b9bac-90e5-422f-9f3a-02e2f73f6cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.641865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b9bac-90e5-422f-9f3a-02e2f73f6cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.642511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b9bac-90e5-422f-9f3a-02e2f73f6cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:05.644410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b9bac-90e5-422f-9f3a-02e2f73f6cc6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:05.675676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1071/2144] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:30:05 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:30:05.679837 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b9bac-90e5-422f-9f3a-02e2f73f6cc6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:05.680260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e59aba2f-64cd-403d-a8b8-69c42c134bed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39cc778e84014ed98f33554b7c46b174', 'request_id': 'req-e59aba2f-64cd-403d-a8b8-69c42c134bed'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.690206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e59aba2f-64cd-403d-a8b8-69c42c134bed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39cc778e84014ed98f33554b7c46b174', 'request_id': 'req-e59aba2f-64cd-403d-a8b8-69c42c134bed'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.696457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e59aba2f-64cd-403d-a8b8-69c42c134bed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39cc778e84014ed98f33554b7c46b174', 'request_id': 'req-e59aba2f-64cd-403d-a8b8-69c42c134bed'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.707960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e59aba2f-64cd-403d-a8b8-69c42c134bed admin admin] The token cache is being invalidate because project 39cc778e84014ed98f33554b7c46b174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:30:05.708253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e59aba2f-64cd-403d-a8b8-69c42c134bed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.708999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e59aba2f-64cd-403d-a8b8-69c42c134bed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:05.710971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1074/2145] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:30:05 2022] DELETE /identity/v3/projects/39cc778e84014ed98f33554b7c46b174 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:05.715719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de127c-ac01-4c98-b9db-3687c08d4eb0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.716026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de127c-ac01-4c98-b9db-3687c08d4eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.716436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de127c-ac01-4c98-b9db-3687c08d4eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.735896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b1de127c-ac01-4c98-b9db-3687c08d4eb0 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:05.737004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b1de127c-ac01-4c98-b9db-3687c08d4eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:05.743176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b9bac-90e5-422f-9f3a-02e2f73f6cc6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:05.744760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1075/2146] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:30:05.747981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf18a4e-e6d6-4013-9f25-b831dec35ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:05.748590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf18a4e-e6d6-4013-9f25-b831dec35ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.748870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf18a4e-e6d6-4013-9f25-b831dec35ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.749174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf18a4e-e6d6-4013-9f25-b831dec35ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.749805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf18a4e-e6d6-4013-9f25-b831dec35ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:05.751835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf18a4e-e6d6-4013-9f25-b831dec35ec3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:05.752361 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c666456-b75a-459d-97c3-58bf80d2c8de None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.752643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c666456-b75a-459d-97c3-58bf80d2c8de None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.752919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c666456-b75a-459d-97c3-58bf80d2c8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.764228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:05.772152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9c666456-b75a-459d-97c3-58bf80d2c8de None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:05.773416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9c666456-b75a-459d-97c3-58bf80d2c8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:05.778509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1072/2147] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:30:05 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:30:05.781780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1076/2148] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:30:05.783936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf18a4e-e6d6-4013-9f25-b831dec35ec3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:05.785556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c666456-b75a-459d-97c3-58bf80d2c8de None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:05.785780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0bf18a4e-e6d6-4013-9f25-b831dec35ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:05.794477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c666456-b75a-459d-97c3-58bf80d2c8de None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:05.797775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf18a4e-e6d6-4013-9f25-b831dec35ec3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:05.799784 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b52d28-2b10-42ea-a93b-f3c0f419b78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:05.800485 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52d28-2b10-42ea-a93b-f3c0f419b78e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.800859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52d28-2b10-42ea-a93b-f3c0f419b78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.801300 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52d28-2b10-42ea-a93b-f3c0f419b78e admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.802039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b52d28-2b10-42ea-a93b-f3c0f419b78e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:30:05.802443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b52d28-2b10-42ea-a93b-f3c0f419b78e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:05.803135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdc377-1753-4269-bd9e-fcffd49aa932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:05.803738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdc377-1753-4269-bd9e-fcffd49aa932 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.804020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdc377-1753-4269-bd9e-fcffd49aa932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.804145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b52d28-2b10-42ea-a93b-f3c0f419b78e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:05.804291 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdc377-1753-4269-bd9e-fcffd49aa932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.804933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdc377-1753-4269-bd9e-fcffd49aa932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:05.806545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdc377-1753-4269-bd9e-fcffd49aa932 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:05.819324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1077/2149] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:30:05.819518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1073/2150] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:30:05.821174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b52d28-2b10-42ea-a93b-f3c0f419b78e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:05.821285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdc377-1753-4269-bd9e-fcffd49aa932 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:05.830975 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdc377-1753-4269-bd9e-fcffd49aa932 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:05.831123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b52d28-2b10-42ea-a93b-f3c0f419b78e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:05.835730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053742c-c2ad-4c27-abb7-1c6bf9c8c11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:05.835914 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6863fa-7140-4247-a4f4-8f6b33f86e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:05.836315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053742c-c2ad-4c27-abb7-1c6bf9c8c11a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.836501 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6863fa-7140-4247-a4f4-8f6b33f86e86 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.836590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053742c-c2ad-4c27-abb7-1c6bf9c8c11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.836782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6863fa-7140-4247-a4f4-8f6b33f86e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.836887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053742c-c2ad-4c27-abb7-1c6bf9c8c11a admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.837058 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6863fa-7140-4247-a4f4-8f6b33f86e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.837510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053742c-c2ad-4c27-abb7-1c6bf9c8c11a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:05.837705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6863fa-7140-4247-a4f4-8f6b33f86e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:05.839235 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053742c-c2ad-4c27-abb7-1c6bf9c8c11a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:05.839572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6863fa-7140-4247-a4f4-8f6b33f86e86 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:05.853731 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1074/2151] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:30:05.854696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053742c-c2ad-4c27-abb7-1c6bf9c8c11a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:05.865352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053742c-c2ad-4c27-abb7-1c6bf9c8c11a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:05.869911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fab032-af13-4a0a-af47-7b4dbebdd916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:05.870311 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1078/2152] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:30:05 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:30:05.870554 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fab032-af13-4a0a-af47-7b4dbebdd916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.870833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fab032-af13-4a0a-af47-7b4dbebdd916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.871105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fab032-af13-4a0a-af47-7b4dbebdd916 admin admin] PATH_INFO: `/v3/projects/7db3c3282ab64e6c8fc13ef85a787833/users/e99b1ae919154f03b2a186e444b6d52e/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.874227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6863fa-7140-4247-a4f4-8f6b33f86e86 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:05.885771 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fab032-af13-4a0a-af47-7b4dbebdd916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7db3c3282ab64e6c8fc13ef85a787833, user_id=e99b1ae919154f03b2a186e444b6d52e, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:05.888764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fab032-af13-4a0a-af47-7b4dbebdd916 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:05.905303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1075/2153] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:30:05 2022] PUT /identity/v3/projects/7db3c3282ab64e6c8fc13ef85a787833/users/e99b1ae919154f03b2a186e444b6d52e/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:05.915742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cde87-2b58-4467-a720-a649688b6e58 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:05.916032 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cde87-2b58-4467-a720-a649688b6e58 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:05.916317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cde87-2b58-4467-a720-a649688b6e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:05.944415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0b3cde87-2b58-4467-a720-a649688b6e58 None None] MFA Rules not processed for user `e99b1ae919154f03b2a186e444b6d52e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:05.945652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0b3cde87-2b58-4467-a720-a649688b6e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:05.947046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6863fa-7140-4247-a4f4-8f6b33f86e86 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:05.965880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:06.007344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631dabb-b1a7-47d3-8e1f-232b9c9fb87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:06.007871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631dabb-b1a7-47d3-8e1f-232b9c9fb87e admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:06.008148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631dabb-b1a7-47d3-8e1f-232b9c9fb87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:06.008417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631dabb-b1a7-47d3-8e1f-232b9c9fb87e admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:06.009128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631dabb-b1a7-47d3-8e1f-232b9c9fb87e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:06.010507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1076/2154] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:30:06.011022 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631dabb-b1a7-47d3-8e1f-232b9c9fb87e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:06.021747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cde87-2b58-4467-a720-a649688b6e58 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:06.023405 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0b3cde87-2b58-4467-a720-a649688b6e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:06.035892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1079/2155] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:30:05 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:30:06.039920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631dabb-b1a7-47d3-8e1f-232b9c9fb87e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:06.049713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631dabb-b1a7-47d3-8e1f-232b9c9fb87e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:06.054752 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad3d17-c28e-4d29-abd8-171aba356adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:06.055339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad3d17-c28e-4d29-abd8-171aba356adf admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:06.055609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad3d17-c28e-4d29-abd8-171aba356adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:06.055871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad3d17-c28e-4d29-abd8-171aba356adf admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:06.056500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad3d17-c28e-4d29-abd8-171aba356adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:06.058135 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad3d17-c28e-4d29-abd8-171aba356adf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:06.069715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1080/2156] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:30:06.073743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad3d17-c28e-4d29-abd8-171aba356adf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:06.081369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cde87-2b58-4467-a720-a649688b6e58 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:06.083305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad3d17-c28e-4d29-abd8-171aba356adf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:06.084939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3120bb-9b6a-4a6d-b836-d853925fcd29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:06.085587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3120bb-9b6a-4a6d-b836-d853925fcd29 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:06.085870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3120bb-9b6a-4a6d-b836-d853925fcd29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:06.086141 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3120bb-9b6a-4a6d-b836-d853925fcd29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:06.087847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e643c3f-c436-457c-b19a-a0e4bb96bb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:06.088428 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e643c3f-c436-457c-b19a-a0e4bb96bb7a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:06.088707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e643c3f-c436-457c-b19a-a0e4bb96bb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:06.088972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e643c3f-c436-457c-b19a-a0e4bb96bb7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:06.089635 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e643c3f-c436-457c-b19a-a0e4bb96bb7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:06.091223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e643c3f-c436-457c-b19a-a0e4bb96bb7a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:06.094471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3120bb-9b6a-4a6d-b836-d853925fcd29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:06.095953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3120bb-9b6a-4a6d-b836-d853925fcd29 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:06.102475 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1081/2157] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:30:06.105755 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1077/2158] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:30:06.106292 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e643c3f-c436-457c-b19a-a0e4bb96bb7a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:06.115658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e643c3f-c436-457c-b19a-a0e4bb96bb7a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:06.120211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957d730-0870-40ba-918f-8d52a750a1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:06.120849 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957d730-0870-40ba-918f-8d52a750a1e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:06.121151 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957d730-0870-40ba-918f-8d52a750a1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:06.121426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957d730-0870-40ba-918f-8d52a750a1e5 admin admin] PATH_INFO: `/v3/projects/d54f5f66e0eb4be9aad2e6d37e0b6db6/users/5bb515269d1a45f594f3d297c7fd267e/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:06.135654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957d730-0870-40ba-918f-8d52a750a1e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54f5f66e0eb4be9aad2e6d37e0b6db6, user_id=5bb515269d1a45f594f3d297c7fd267e, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:06.138275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957d730-0870-40ba-918f-8d52a750a1e5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:06.153771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1082/2159] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:30:06 2022] PUT /identity/v3/projects/d54f5f66e0eb4be9aad2e6d37e0b6db6/users/5bb515269d1a45f594f3d297c7fd267e/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:06.159099 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d8743-7bb0-455c-9d29-83b791be4107 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:06.160129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d8743-7bb0-455c-9d29-83b791be4107 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:06.160472 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d8743-7bb0-455c-9d29-83b791be4107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:06.182102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0f0d8743-7bb0-455c-9d29-83b791be4107 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:06.183369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0f0d8743-7bb0-455c-9d29-83b791be4107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:06.227778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:06.266924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1078/2160] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:06.276550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957d730-0870-40ba-918f-8d52a750a1e5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:06.278402 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1957d730-0870-40ba-918f-8d52a750a1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:06.335022 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957d730-0870-40ba-918f-8d52a750a1e5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:06.338955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b38c86-475e-47f4-8e64-835ef96c28d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:06.339597 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b38c86-475e-47f4-8e64-835ef96c28d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:06.339873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b38c86-475e-47f4-8e64-835ef96c28d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:06.340137 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b38c86-475e-47f4-8e64-835ef96c28d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:06.349440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b38c86-475e-47f4-8e64-835ef96c28d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:06.351262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b38c86-475e-47f4-8e64-835ef96c28d6 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:06.363209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1083/2161] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:30:06.821374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d8743-7bb0-455c-9d29-83b791be4107 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:06.822976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0f0d8743-7bb0-455c-9d29-83b791be4107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:06.881317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d8743-7bb0-455c-9d29-83b791be4107 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:06.885139 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd95897-36bb-4bda-a95b-ee86fbc052d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:06.885910 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd95897-36bb-4bda-a95b-ee86fbc052d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:06.886189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd95897-36bb-4bda-a95b-ee86fbc052d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:06.886459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd95897-36bb-4bda-a95b-ee86fbc052d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:06.888231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6fd95897-36bb-4bda-a95b-ee86fbc052d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:06.961860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd95897-36bb-4bda-a95b-ee86fbc052d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:06.963500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd95897-36bb-4bda-a95b-ee86fbc052d0 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:06.973721 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1079/2162] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:30:08.843849 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32679a77-2668-41a1-a656-6019bed4b466 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:08.845258 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32679a77-2668-41a1-a656-6019bed4b466 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:08.845687 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32679a77-2668-41a1-a656-6019bed4b466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:08.884493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-32679a77-2668-41a1-a656-6019bed4b466 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:08.885819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-32679a77-2668-41a1-a656-6019bed4b466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:08.979266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1084/2163] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:30:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:08.980213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd95897-36bb-4bda-a95b-ee86fbc052d0 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:08.982392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6fd95897-36bb-4bda-a95b-ee86fbc052d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:09.108971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd95897-36bb-4bda-a95b-ee86fbc052d0 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:09.128548 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8c68b-740c-4b3b-bc72-28f9807004b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:09.129230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8c68b-740c-4b3b-bc72-28f9807004b7 service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:09.129507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8c68b-740c-4b3b-bc72-28f9807004b7 service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:09.130353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8c68b-740c-4b3b-bc72-28f9807004b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:09.155029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8c68b-740c-4b3b-bc72-28f9807004b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:09.158346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8c68b-740c-4b3b-bc72-28f9807004b7 service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:09.195251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1080/2164] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:30:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 04:30:12.994953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58f19c-409e-45a1-bb5b-d2b252c4f55f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:12.995521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58f19c-409e-45a1-bb5b-d2b252c4f55f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:12.995521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58f19c-409e-45a1-bb5b-d2b252c4f55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:13.020857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:13.024610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bd58f19c-409e-45a1-bb5b-d2b252c4f55f None None] MFA Rules not processed for user `5bb515269d1a45f594f3d297c7fd267e`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:13.025990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bd58f19c-409e-45a1-bb5b-d2b252c4f55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:13.091990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1085/2165] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:13.103017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0407bb-07e8-493e-b716-a72ee80f721d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:13.103525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0407bb-07e8-493e-b716-a72ee80f721d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:13.103623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0407bb-07e8-493e-b716-a72ee80f721d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:13.123156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0e0407bb-07e8-493e-b716-a72ee80f721d None None] MFA Rules not processed for user `5bb515269d1a45f594f3d297c7fd267e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:13.124266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0e0407bb-07e8-493e-b716-a72ee80f721d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:13.150096 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1081/2166] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:13.170075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58f19c-409e-45a1-bb5b-d2b252c4f55f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:13.171754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bd58f19c-409e-45a1-bb5b-d2b252c4f55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:13.381116 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58f19c-409e-45a1-bb5b-d2b252c4f55f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:13.392582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61feed43-6c55-47b3-adaa-d4cc3b90e7e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:13.393163 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61feed43-6c55-47b3-adaa-d4cc3b90e7e6 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:13.393445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61feed43-6c55-47b3-adaa-d4cc3b90e7e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:13.393716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61feed43-6c55-47b3-adaa-d4cc3b90e7e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:13.407952 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61feed43-6c55-47b3-adaa-d4cc3b90e7e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:13.410004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61feed43-6c55-47b3-adaa-d4cc3b90e7e6 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:13.446712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1086/2167] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:30:14.468522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594235a-f8eb-43e0-90ea-8bda81423617 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:14.469173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594235a-f8eb-43e0-90ea-8bda81423617 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:14.469173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594235a-f8eb-43e0-90ea-8bda81423617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:14.493659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2594235a-f8eb-43e0-90ea-8bda81423617 None None] MFA Rules not processed for user `5bb515269d1a45f594f3d297c7fd267e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:14.495215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2594235a-f8eb-43e0-90ea-8bda81423617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:14.508027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1082/2168] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:19.570193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61feed43-6c55-47b3-adaa-d4cc3b90e7e6 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:19.571615 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-61feed43-6c55-47b3-adaa-d4cc3b90e7e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:19.592450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61feed43-6c55-47b3-adaa-d4cc3b90e7e6 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:19.603641 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eda9aa-9c0d-4b33-97c6-43125d5b9ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:19.604364 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eda9aa-9c0d-4b33-97c6-43125d5b9ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:19.604640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eda9aa-9c0d-4b33-97c6-43125d5b9ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:19.606376 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eda9aa-9c0d-4b33-97c6-43125d5b9ef5 admin admin] PATH_INFO: `/v3/users/5cd4c6d9f4a746f982506e2898268ad5` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:19.608708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eda9aa-9c0d-4b33-97c6-43125d5b9ef5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cd4c6d9f4a746f982506e2898268ad5)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:19.611126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eda9aa-9c0d-4b33-97c6-43125d5b9ef5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:19.622537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594235a-f8eb-43e0-90ea-8bda81423617 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:19.624369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2594235a-f8eb-43e0-90ea-8bda81423617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:19.644113 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594235a-f8eb-43e0-90ea-8bda81423617 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:19.654441 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454c8ee-9d88-4ff3-95b0-3dcf6be4c467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:19.655422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454c8ee-9d88-4ff3-95b0-3dcf6be4c467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:19.655700 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454c8ee-9d88-4ff3-95b0-3dcf6be4c467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:19.656137 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454c8ee-9d88-4ff3-95b0-3dcf6be4c467 admin admin] PATH_INFO: `/v3/users/6a1834aabab24c2694ab7f7dbab936a4` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:19.657186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454c8ee-9d88-4ff3-95b0-3dcf6be4c467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a1834aabab24c2694ab7f7dbab936a4)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:19.677509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454c8ee-9d88-4ff3-95b0-3dcf6be4c467 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:19.682781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:19.705756 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-85eda9aa-9c0d-4b33-97c6-43125d5b9ef5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cd4c6d9f4a746f982506e2898268ad5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:19.722827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-85eda9aa-9c0d-4b33-97c6-43125d5b9ef5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cd4c6d9f4a746f982506e2898268ad5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:19.723467 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-85eda9aa-9c0d-4b33-97c6-43125d5b9ef5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cd4c6d9f4a746f982506e2898268ad5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:19.734925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-85eda9aa-9c0d-4b33-97c6-43125d5b9ef5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cd4c6d9f4a746f982506e2898268ad5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:19.778266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-85eda9aa-9c0d-4b33-97c6-43125d5b9ef5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cd4c6d9f4a746f982506e2898268ad5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:19.779664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9454c8ee-9d88-4ff3-95b0-3dcf6be4c467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a1834aabab24c2694ab7f7dbab936a4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:19.787297 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:19.797595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9454c8ee-9d88-4ff3-95b0-3dcf6be4c467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a1834aabab24c2694ab7f7dbab936a4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:19.800973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9454c8ee-9d88-4ff3-95b0-3dcf6be4c467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a1834aabab24c2694ab7f7dbab936a4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:19.804323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1087/2169] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:30:19 2022] DELETE /identity/v3/users/5cd4c6d9f4a746f982506e2898268ad5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:19.809180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a1264-8336-46c8-92dd-6f920e7ff580 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:19.809471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a1264-8336-46c8-92dd-6f920e7ff580 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:19.809758 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a1264-8336-46c8-92dd-6f920e7ff580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:19.816578 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9454c8ee-9d88-4ff3-95b0-3dcf6be4c467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a1834aabab24c2694ab7f7dbab936a4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:19.861074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9454c8ee-9d88-4ff3-95b0-3dcf6be4c467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a1834aabab24c2694ab7f7dbab936a4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:19.873770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-404a1264-8336-46c8-92dd-6f920e7ff580 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:19.875153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-404a1264-8336-46c8-92dd-6f920e7ff580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:19.877965 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1083/2170] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:30:19 2022] DELETE /identity/v3/users/6a1834aabab24c2694ab7f7dbab936a4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:19.883075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6944f83-731f-44d1-a3b5-8b5283ec0f47 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:19.883364 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6944f83-731f-44d1-a3b5-8b5283ec0f47 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:19.883633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6944f83-731f-44d1-a3b5-8b5283ec0f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:19.919570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f6944f83-731f-44d1-a3b5-8b5283ec0f47 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:19.920791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f6944f83-731f-44d1-a3b5-8b5283ec0f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:19.970024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1088/2171] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:19.984755 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a1264-8336-46c8-92dd-6f920e7ff580 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:19.987925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-404a1264-8336-46c8-92dd-6f920e7ff580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:20.011574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1084/2172] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:30:20.023708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6944f83-731f-44d1-a3b5-8b5283ec0f47 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:20.025613 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f6944f83-731f-44d1-a3b5-8b5283ec0f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:20.061426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a1264-8336-46c8-92dd-6f920e7ff580 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:20.065013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb345a0d-48e5-4986-b81c-05e28b6dd3d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:20.065663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb345a0d-48e5-4986-b81c-05e28b6dd3d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:20.065936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb345a0d-48e5-4986-b81c-05e28b6dd3d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:20.066202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb345a0d-48e5-4986-b81c-05e28b6dd3d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:20.075094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb345a0d-48e5-4986-b81c-05e28b6dd3d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:20.076684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb345a0d-48e5-4986-b81c-05e28b6dd3d4 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:20.083847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6944f83-731f-44d1-a3b5-8b5283ec0f47 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:20.086581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1089/2173] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:30:20.087439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d09a30-3020-4b42-8ed7-bc2bee57d28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:20.088046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d09a30-3020-4b42-8ed7-bc2bee57d28a service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:20.088318 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d09a30-3020-4b42-8ed7-bc2bee57d28a service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:20.088586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d09a30-3020-4b42-8ed7-bc2bee57d28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:20.097713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d09a30-3020-4b42-8ed7-bc2bee57d28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:20.099123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d09a30-3020-4b42-8ed7-bc2bee57d28a service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:20.109547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1085/2174] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:30:20.397072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb345a0d-48e5-4986-b81c-05e28b6dd3d4 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:20.398724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bb345a0d-48e5-4986-b81c-05e28b6dd3d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:20.429678 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb345a0d-48e5-4986-b81c-05e28b6dd3d4 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:20.438597 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:20.439266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:20.439547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:20.439780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] PATH_INFO: `/v3/projects/959d8cd72ac744628842d835f315c75d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:20.441187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=959d8cd72ac744628842d835f315c75d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:20.446086 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:20.457371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d09a30-3020-4b42-8ed7-bc2bee57d28a service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:20.459890 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c9d09a30-3020-4b42-8ed7-bc2bee57d28a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:20.487759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d09a30-3020-4b42-8ed7-bc2bee57d28a service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:20.493352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:20.494764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:20.495051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:20.495342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] PATH_INFO: `/v3/projects/f8bb6374238e40688486fe4a26879f65` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:20.496523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8bb6374238e40688486fe4a26879f65)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:20.498511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:20.545282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '959d8cd72ac744628842d835f315c75d', 'request_id': 'req-195da40a-04e6-4522-97e7-b1364d3921d7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.546865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:20.564107 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '959d8cd72ac744628842d835f315c75d', 'request_id': 'req-195da40a-04e6-4522-97e7-b1364d3921d7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.577894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '959d8cd72ac744628842d835f315c75d', 'request_id': 'req-195da40a-04e6-4522-97e7-b1364d3921d7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.607854 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] The token cache is being invalidate because project 959d8cd72ac744628842d835f315c75d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:30:20.608309 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.609242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.619061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8bb6374238e40688486fe4a26879f65', 'request_id': 'req-a91e4de3-8607-47f2-8a8c-80c3545e4dad'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.619768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1090/2175] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:30:20 2022] DELETE /identity/v3/projects/959d8cd72ac744628842d835f315c75d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:20.623827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:20.630026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:20.643075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8bb6374238e40688486fe4a26879f65', 'request_id': 'req-a91e4de3-8607-47f2-8a8c-80c3545e4dad'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.668787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8bb6374238e40688486fe4a26879f65', 'request_id': 'req-a91e4de3-8607-47f2-8a8c-80c3545e4dad'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.681776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] The token cache is being invalidate because project f8bb6374238e40688486fe4a26879f65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:30:20.682062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.682751 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.684274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1086/2176] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:30:20 2022] DELETE /identity/v3/projects/f8bb6374238e40688486fe4a26879f65 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:20.684393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:20.688196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:20.689708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:20.726296 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195da40a-04e6-4522-97e7-b1364d3921d7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:20.755529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e4de3-8607-47f2-8a8c-80c3545e4dad admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:20.760160 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:20.760823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:20.761123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:20.761401 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] PATH_INFO: `/v3/users/d9d404df955b4d8283139f3ad12b9880` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:20.762543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9d404df955b4d8283139f3ad12b9880)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:20.764452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:20.780945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:20.781725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:20.782033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:20.782365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] PATH_INFO: `/v3/users/19287ee340be4e3aa62645fc4eb4a81e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:20.783619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=19287ee340be4e3aa62645fc4eb4a81e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:20.785649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:20.815674 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9d404df955b4d8283139f3ad12b9880'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.823415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9d404df955b4d8283139f3ad12b9880'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.824107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9d404df955b4d8283139f3ad12b9880'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.831411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9d404df955b4d8283139f3ad12b9880'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.839948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19287ee340be4e3aa62645fc4eb4a81e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.849542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19287ee340be4e3aa62645fc4eb4a81e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.849962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9d404df955b4d8283139f3ad12b9880'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.850743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19287ee340be4e3aa62645fc4eb4a81e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.862516 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1087/2177] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:30:20 2022] DELETE /identity/v3/users/d9d404df955b4d8283139f3ad12b9880 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:20.879949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19287ee340be4e3aa62645fc4eb4a81e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.910603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19287ee340be4e3aa62645fc4eb4a81e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:20.924990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1091/2178] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:30:20 2022] DELETE /identity/v3/users/19287ee340be4e3aa62645fc4eb4a81e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:21.456427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:21.457380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:21.477530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:21.477530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:21.523711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:21.583448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7c6da-8f35-4caa-b4cc-f2390e7bca25 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:21.592797 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:21.620901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc0c01-7575-4111-a49d-7ac7c7323b5a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:21.633486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effd556-9211-44a2-8d8d-f2d619f08d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:21.634113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effd556-9211-44a2-8d8d-f2d619f08d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:21.634390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effd556-9211-44a2-8d8d-f2d619f08d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:21.634729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effd556-9211-44a2-8d8d-f2d619f08d85 admin admin] PATH_INFO: `/v3/projects/ff018089532145a3aadc4bdc0f480ae1` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:21.636130 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effd556-9211-44a2-8d8d-f2d619f08d85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff018089532145a3aadc4bdc0f480ae1)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:21.639369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effd556-9211-44a2-8d8d-f2d619f08d85 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:21.692038 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:21.692957 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:21.692957 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:21.693730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] PATH_INFO: `/v3/projects/872071de909b45c096dc64279bdf7205` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:21.694716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=872071de909b45c096dc64279bdf7205)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:21.696746 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:21.724707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9effd556-9211-44a2-8d8d-f2d619f08d85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff018089532145a3aadc4bdc0f480ae1', 'request_id': 'req-9effd556-9211-44a2-8d8d-f2d619f08d85'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.738295 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9effd556-9211-44a2-8d8d-f2d619f08d85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff018089532145a3aadc4bdc0f480ae1', 'request_id': 'req-9effd556-9211-44a2-8d8d-f2d619f08d85'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.745509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9effd556-9211-44a2-8d8d-f2d619f08d85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff018089532145a3aadc4bdc0f480ae1', 'request_id': 'req-9effd556-9211-44a2-8d8d-f2d619f08d85'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.758492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9effd556-9211-44a2-8d8d-f2d619f08d85 admin admin] The token cache is being invalidate because project ff018089532145a3aadc4bdc0f480ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:30:21.758783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9effd556-9211-44a2-8d8d-f2d619f08d85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.759890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9effd556-9211-44a2-8d8d-f2d619f08d85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.765560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1092/2179] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:30:21 2022] DELETE /identity/v3/projects/ff018089532145a3aadc4bdc0f480ae1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:21.767402 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87835d45-0d77-45e9-9cee-bd16abaf200c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:21.767751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87835d45-0d77-45e9-9cee-bd16abaf200c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:21.767963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87835d45-0d77-45e9-9cee-bd16abaf200c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:21.788406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '872071de909b45c096dc64279bdf7205', 'request_id': 'req-97305688-6e6d-4c3b-87fe-e3544717b871'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.790978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-87835d45-0d77-45e9-9cee-bd16abaf200c None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:21.792183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-87835d45-0d77-45e9-9cee-bd16abaf200c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:21.799579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '872071de909b45c096dc64279bdf7205', 'request_id': 'req-97305688-6e6d-4c3b-87fe-e3544717b871'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.806439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '872071de909b45c096dc64279bdf7205', 'request_id': 'req-97305688-6e6d-4c3b-87fe-e3544717b871'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.820935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] The token cache is being invalidate because project 872071de909b45c096dc64279bdf7205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:30:21.821274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.824431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.829031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1088/2180] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:30:21 2022] DELETE /identity/v3/projects/872071de909b45c096dc64279bdf7205 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:21.830127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:21.831675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:21.859051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1093/2181] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:21.866923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bab54-b537-4a14-8e55-460850518901 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:21.867226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bab54-b537-4a14-8e55-460850518901 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:21.867508 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bab54-b537-4a14-8e55-460850518901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:21.886540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ec7bab54-b537-4a14-8e55-460850518901 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:21.888023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ec7bab54-b537-4a14-8e55-460850518901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:21.897313 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1094/2182] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:30:21.901360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bab54-b537-4a14-8e55-460850518901 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:21.902214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97305688-6e6d-4c3b-87fe-e3544717b871 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:21.907169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:21.907812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:21.908100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:21.908374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] PATH_INFO: `/v3/users/cb9fe17b7a8c4c13bc7f2f1cc8b663c8` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:21.910860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb9fe17b7a8c4c13bc7f2f1cc8b663c8)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:21.910996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bab54-b537-4a14-8e55-460850518901 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:21.911997 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:21.915723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab28911-c533-4f8a-bf4b-304847a6170b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:21.916315 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab28911-c533-4f8a-bf4b-304847a6170b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:21.916660 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab28911-c533-4f8a-bf4b-304847a6170b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:21.917000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab28911-c533-4f8a-bf4b-304847a6170b admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:21.917526 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:21.917755 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab28911-c533-4f8a-bf4b-304847a6170b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:30:21.917976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab28911-c533-4f8a-bf4b-304847a6170b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:21.919800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab28911-c533-4f8a-bf4b-304847a6170b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:21.937262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1095/2183] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:30:21.939874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab28911-c533-4f8a-bf4b-304847a6170b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:21.952250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab28911-c533-4f8a-bf4b-304847a6170b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:21.959098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16db684-2c22-4f93-9858-fde9e1b42e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:21.959748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16db684-2c22-4f93-9858-fde9e1b42e0b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:21.960033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16db684-2c22-4f93-9858-fde9e1b42e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:21.960300 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16db684-2c22-4f93-9858-fde9e1b42e0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:21.960939 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16db684-2c22-4f93-9858-fde9e1b42e0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:21.962945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16db684-2c22-4f93-9858-fde9e1b42e0b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:21.973819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb9fe17b7a8c4c13bc7f2f1cc8b663c8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.982203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb9fe17b7a8c4c13bc7f2f1cc8b663c8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.982959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb9fe17b7a8c4c13bc7f2f1cc8b663c8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:21.990985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb9fe17b7a8c4c13bc7f2f1cc8b663c8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:22.000818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1096/2184] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:30:21 2022] POST /identity/v3/projects => generated 357 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:30:22.005100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16db684-2c22-4f93-9858-fde9e1b42e0b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.005803 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb9fe17b7a8c4c13bc7f2f1cc8b663c8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:22.006719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a16db684-2c22-4f93-9858-fde9e1b42e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:22.012204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1089/2185] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:30:21 2022] DELETE /identity/v3/users/cb9fe17b7a8c4c13bc7f2f1cc8b663c8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:22.056474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:22.080511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16db684-2c22-4f93-9858-fde9e1b42e0b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.139736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f851c-7c24-4025-929e-a51d09ffb631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.140349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f851c-7c24-4025-929e-a51d09ffb631 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.140626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f851c-7c24-4025-929e-a51d09ffb631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.140896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f851c-7c24-4025-929e-a51d09ffb631 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.141598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f851c-7c24-4025-929e-a51d09ffb631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.143529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f851c-7c24-4025-929e-a51d09ffb631 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.168663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1097/2186] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:30:22 2022] POST /identity/v3/users => generated 279 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:30:22.172579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.183595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0adf69-fa75-4457-a3da-f5c6309ee51a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.190673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e2616-26dd-486d-95f8-05cdade60c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.191311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e2616-26dd-486d-95f8-05cdade60c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.192071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e2616-26dd-486d-95f8-05cdade60c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.192071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e2616-26dd-486d-95f8-05cdade60c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.192583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e2616-26dd-486d-95f8-05cdade60c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.194244 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e2616-26dd-486d-95f8-05cdade60c99 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.206858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1090/2187] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:30:22.217578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f851c-7c24-4025-929e-a51d09ffb631 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.228572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f851c-7c24-4025-929e-a51d09ffb631 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.233985 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414d6de-f993-4c61-b422-751886bf7f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.234606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414d6de-f993-4c61-b422-751886bf7f4e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.234884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414d6de-f993-4c61-b422-751886bf7f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.235485 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414d6de-f993-4c61-b422-751886bf7f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.235851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414d6de-f993-4c61-b422-751886bf7f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.237495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414d6de-f993-4c61-b422-751886bf7f4e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.253910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1098/2188] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:30:22.258383 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e2616-26dd-486d-95f8-05cdade60c99 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.281834 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e2616-26dd-486d-95f8-05cdade60c99 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.295941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446578c-d997-4e44-a4e1-94f5bc4c79fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.296781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446578c-d997-4e44-a4e1-94f5bc4c79fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.299449 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446578c-d997-4e44-a4e1-94f5bc4c79fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.301598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446578c-d997-4e44-a4e1-94f5bc4c79fa admin admin] PATH_INFO: `/v3/projects/2fdd592ec7c042b28c573479dc08d3b7/users/85284acd929d4042ba6c42454aa68e70/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.322050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446578c-d997-4e44-a4e1-94f5bc4c79fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fdd592ec7c042b28c573479dc08d3b7, user_id=85284acd929d4042ba6c42454aa68e70, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.327009 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446578c-d997-4e44-a4e1-94f5bc4c79fa admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.337238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414d6de-f993-4c61-b422-751886bf7f4e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.339302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d414d6de-f993-4c61-b422-751886bf7f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:22.357368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1091/2189] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:30:22 2022] PUT /identity/v3/projects/2fdd592ec7c042b28c573479dc08d3b7/users/85284acd929d4042ba6c42454aa68e70/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:22.362396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a666e7-f1bb-443b-ad92-f45e4e41c8bc None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.362636 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a666e7-f1bb-443b-ad92-f45e4e41c8bc None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.363479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a666e7-f1bb-443b-ad92-f45e4e41c8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.375835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414d6de-f993-4c61-b422-751886bf7f4e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.403081 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-03a666e7-f1bb-443b-ad92-f45e4e41c8bc None None] MFA Rules not processed for user `85284acd929d4042ba6c42454aa68e70`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:22.404885 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-03a666e7-f1bb-443b-ad92-f45e4e41c8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:22.488416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1092/2190] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:30:22.492960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a666e7-f1bb-443b-ad92-f45e4e41c8bc None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.496004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff39a49-0c48-4722-a72c-478c82810c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.496662 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff39a49-0c48-4722-a72c-478c82810c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.496971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff39a49-0c48-4722-a72c-478c82810c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.497277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff39a49-0c48-4722-a72c-478c82810c32 admin admin] PATH_INFO: `/v3/projects/d33c55d1763e457392c5b504e5aff70f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.498421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff39a49-0c48-4722-a72c-478c82810c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d33c55d1763e457392c5b504e5aff70f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.500340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff39a49-0c48-4722-a72c-478c82810c32 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.503448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a666e7-f1bb-443b-ad92-f45e4e41c8bc None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.508217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78952e81-1b6d-4bc3-a7c8-aab0c0971ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.508812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78952e81-1b6d-4bc3-a7c8-aab0c0971ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.509127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78952e81-1b6d-4bc3-a7c8-aab0c0971ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.509409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78952e81-1b6d-4bc3-a7c8-aab0c0971ca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.510036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78952e81-1b6d-4bc3-a7c8-aab0c0971ca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.512005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78952e81-1b6d-4bc3-a7c8-aab0c0971ca2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.541596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1093/2191] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:30:22 2022] POST /identity/v3/projects => generated 355 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:30:22.545226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78952e81-1b6d-4bc3-a7c8-aab0c0971ca2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.553773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3ff39a49-0c48-4722-a72c-478c82810c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd33c55d1763e457392c5b504e5aff70f', 'request_id': 'req-3ff39a49-0c48-4722-a72c-478c82810c32'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:22.561887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3ff39a49-0c48-4722-a72c-478c82810c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd33c55d1763e457392c5b504e5aff70f', 'request_id': 'req-3ff39a49-0c48-4722-a72c-478c82810c32'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:22.564805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:22.568403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3ff39a49-0c48-4722-a72c-478c82810c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd33c55d1763e457392c5b504e5aff70f', 'request_id': 'req-3ff39a49-0c48-4722-a72c-478c82810c32'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:22.576609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:22.582185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3ff39a49-0c48-4722-a72c-478c82810c32 admin admin] The token cache is being invalidate because project d33c55d1763e457392c5b504e5aff70f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:30:22.582668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3ff39a49-0c48-4722-a72c-478c82810c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:22.583765 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3ff39a49-0c48-4722-a72c-478c82810c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:22.585759 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1099/2192] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:30:22 2022] DELETE /identity/v3/projects/d33c55d1763e457392c5b504e5aff70f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:22.590228 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a60e5-e6fc-47c2-8513-583f2fef82c3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.590511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a60e5-e6fc-47c2-8513-583f2fef82c3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.590781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a60e5-e6fc-47c2-8513-583f2fef82c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.609925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-498a60e5-e6fc-47c2-8513-583f2fef82c3 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:22.611241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-498a60e5-e6fc-47c2-8513-583f2fef82c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:22.616277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78952e81-1b6d-4bc3-a7c8-aab0c0971ca2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.619721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1100/2193] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:30:22.622563 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd5bc8-8f1b-46a4-bb4e-7b1177eb0aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.623191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd5bc8-8f1b-46a4-bb4e-7b1177eb0aec admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.623461 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd5bc8-8f1b-46a4-bb4e-7b1177eb0aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.623734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd5bc8-8f1b-46a4-bb4e-7b1177eb0aec admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.624365 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd5bc8-8f1b-46a4-bb4e-7b1177eb0aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.626734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd5bc8-8f1b-46a4-bb4e-7b1177eb0aec admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.627534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b8004-6638-4490-b08b-b5e15fcababf None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.627817 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b8004-6638-4490-b08b-b5e15fcababf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.628090 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b8004-6638-4490-b08b-b5e15fcababf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.649545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fa7b8004-6638-4490-b08b-b5e15fcababf None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:22.649772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1094/2194] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:30:22 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:30:22.650983 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fa7b8004-6638-4490-b08b-b5e15fcababf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:22.653671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd5bc8-8f1b-46a4-bb4e-7b1177eb0aec admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.655287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bfdd5bc8-8f1b-46a4-bb4e-7b1177eb0aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:22.664643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1101/2195] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:30:22.668933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b8004-6638-4490-b08b-b5e15fcababf None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.671579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd5bc8-8f1b-46a4-bb4e-7b1177eb0aec admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.676583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea25a5-9958-409d-b8ea-cb37fd92d993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.677219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea25a5-9958-409d-b8ea-cb37fd92d993 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.677502 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea25a5-9958-409d-b8ea-cb37fd92d993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.677774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea25a5-9958-409d-b8ea-cb37fd92d993 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.678420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea25a5-9958-409d-b8ea-cb37fd92d993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.679343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b8004-6638-4490-b08b-b5e15fcababf None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.680019 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea25a5-9958-409d-b8ea-cb37fd92d993 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.684148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecefa4-994b-4400-8f4b-296baa78f1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.684741 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecefa4-994b-4400-8f4b-296baa78f1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.685017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecefa4-994b-4400-8f4b-296baa78f1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.685322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecefa4-994b-4400-8f4b-296baa78f1c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.685974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ecefa4-994b-4400-8f4b-296baa78f1c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:30:22.686267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ecefa4-994b-4400-8f4b-296baa78f1c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.687860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ecefa4-994b-4400-8f4b-296baa78f1c9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.691219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1095/2196] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:30:22.695230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea25a5-9958-409d-b8ea-cb37fd92d993 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.701344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1102/2197] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:30:22.705543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecefa4-994b-4400-8f4b-296baa78f1c9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.708552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea25a5-9958-409d-b8ea-cb37fd92d993 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.713333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8860-a994-4256-af5e-c86724df86cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.714003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8860-a994-4256-af5e-c86724df86cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.714284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8860-a994-4256-af5e-c86724df86cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.714555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8860-a994-4256-af5e-c86724df86cf admin admin] PATH_INFO: `/v3/projects/b955c4e4289548feaeaecc0081760305/users/17e15c47755a4247a57901d71171f5cc/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.715609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecefa4-994b-4400-8f4b-296baa78f1c9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.725749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c378b91-0241-432e-b667-9e2255f68829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.726368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c378b91-0241-432e-b667-9e2255f68829 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.726645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c378b91-0241-432e-b667-9e2255f68829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.726913 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c378b91-0241-432e-b667-9e2255f68829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.727559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c378b91-0241-432e-b667-9e2255f68829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.729506 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c378b91-0241-432e-b667-9e2255f68829 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.731749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c8860-a994-4256-af5e-c86724df86cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b955c4e4289548feaeaecc0081760305, user_id=17e15c47755a4247a57901d71171f5cc, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.734463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c8860-a994-4256-af5e-c86724df86cf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.751983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1096/2198] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:30:22 2022] PUT /identity/v3/projects/b955c4e4289548feaeaecc0081760305/users/17e15c47755a4247a57901d71171f5cc/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:22.756204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8860-a994-4256-af5e-c86724df86cf admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.765019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1103/2199] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:30:22 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:30:22.766929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c378b91-0241-432e-b667-9e2255f68829 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.821613 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8860-a994-4256-af5e-c86724df86cf admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.826376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f997580-2071-48de-a1b1-50bc3afff403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.827031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f997580-2071-48de-a1b1-50bc3afff403 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.827302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f997580-2071-48de-a1b1-50bc3afff403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.827572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f997580-2071-48de-a1b1-50bc3afff403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.828211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f997580-2071-48de-a1b1-50bc3afff403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.829875 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f997580-2071-48de-a1b1-50bc3afff403 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.830922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c378b91-0241-432e-b667-9e2255f68829 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.836037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ac1a6-9b6e-4f7e-b992-54debe145be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.836640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ac1a6-9b6e-4f7e-b992-54debe145be1 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.836917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ac1a6-9b6e-4f7e-b992-54debe145be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.837411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ac1a6-9b6e-4f7e-b992-54debe145be1 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.838036 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ac1a6-9b6e-4f7e-b992-54debe145be1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.839943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ac1a6-9b6e-4f7e-b992-54debe145be1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.840354 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1097/2200] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:30:22.844533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f997580-2071-48de-a1b1-50bc3afff403 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.855484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f997580-2071-48de-a1b1-50bc3afff403 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.860886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02dbbf-25c6-471c-9c55-d6d88c9e3257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.861583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02dbbf-25c6-471c-9c55-d6d88c9e3257 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.861860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02dbbf-25c6-471c-9c55-d6d88c9e3257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.862132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02dbbf-25c6-471c-9c55-d6d88c9e3257 admin admin] PATH_INFO: `/v3/projects/b955c4e4289548feaeaecc0081760305/users/17e15c47755a4247a57901d71171f5cc/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.863456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1104/2201] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:30:22 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:30:22.866659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02dbbf-25c6-471c-9c55-d6d88c9e3257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b955c4e4289548feaeaecc0081760305, user_id=17e15c47755a4247a57901d71171f5cc, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.866789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02dbbf-25c6-471c-9c55-d6d88c9e3257 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.867515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ac1a6-9b6e-4f7e-b992-54debe145be1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.877726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ac1a6-9b6e-4f7e-b992-54debe145be1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.878302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1098/2202] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:30:22 2022] PUT /identity/v3/projects/b955c4e4289548feaeaecc0081760305/users/17e15c47755a4247a57901d71171f5cc/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:22.888480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b27e1b-edd4-40c7-89bb-b53a6a295a91 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.888759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b27e1b-edd4-40c7-89bb-b53a6a295a91 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.889026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b27e1b-edd4-40c7-89bb-b53a6a295a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.916642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a8b27e1b-edd4-40c7-89bb-b53a6a295a91 None None] MFA Rules not processed for user `17e15c47755a4247a57901d71171f5cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:22.917642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a8b27e1b-edd4-40c7-89bb-b53a6a295a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:22.938320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddf8f9-088c-4a10-a6a3-e9f779d9b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.938969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddf8f9-088c-4a10-a6a3-e9f779d9b6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.939248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddf8f9-088c-4a10-a6a3-e9f779d9b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.939506 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddf8f9-088c-4a10-a6a3-e9f779d9b6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.940150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddf8f9-088c-4a10-a6a3-e9f779d9b6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.941791 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddf8f9-088c-4a10-a6a3-e9f779d9b6d5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.954924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1105/2203] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:30:22.955967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddf8f9-088c-4a10-a6a3-e9f779d9b6d5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:22.959756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:22.966403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddf8f9-088c-4a10-a6a3-e9f779d9b6d5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:22.971096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24351456-87c8-4f6f-94a8-f05c8488ce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:22.971705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351456-87c8-4f6f-94a8-f05c8488ce64 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:22.972329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351456-87c8-4f6f-94a8-f05c8488ce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:22.972604 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351456-87c8-4f6f-94a8-f05c8488ce64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:22.973399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24351456-87c8-4f6f-94a8-f05c8488ce64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:22.974999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24351456-87c8-4f6f-94a8-f05c8488ce64 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:22.989219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1106/2204] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:30:22.990831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1099/2205] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:30:22.991873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24351456-87c8-4f6f-94a8-f05c8488ce64 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:23.001990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b27e1b-edd4-40c7-89bb-b53a6a295a91 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:23.003564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a8b27e1b-edd4-40c7-89bb-b53a6a295a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:23.006953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24351456-87c8-4f6f-94a8-f05c8488ce64 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:23.012425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b046cfb-1c8f-4eec-b207-d27261c2f786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:23.013750 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b046cfb-1c8f-4eec-b207-d27261c2f786 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:23.014399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b046cfb-1c8f-4eec-b207-d27261c2f786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:23.014800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b046cfb-1c8f-4eec-b207-d27261c2f786 admin admin] PATH_INFO: `/v3/projects/7be428966f9641d794c310078b177dc5/users/853d9a4f9d28459a82cc79186e233796/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:23.033145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b046cfb-1c8f-4eec-b207-d27261c2f786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be428966f9641d794c310078b177dc5, user_id=853d9a4f9d28459a82cc79186e233796, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:23.035876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b046cfb-1c8f-4eec-b207-d27261c2f786 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:23.046856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:23.052098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1107/2206] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:30:22 2022] PUT /identity/v3/projects/7be428966f9641d794c310078b177dc5/users/853d9a4f9d28459a82cc79186e233796/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:23.063452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26893cc-2f2c-4233-84b0-298f0a4ac1ab None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:23.063748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26893cc-2f2c-4233-84b0-298f0a4ac1ab None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:23.064019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26893cc-2f2c-4233-84b0-298f0a4ac1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:23.084153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b27e1b-edd4-40c7-89bb-b53a6a295a91 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:23.107911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b26893cc-2f2c-4233-84b0-298f0a4ac1ab None None] MFA Rules not processed for user `853d9a4f9d28459a82cc79186e233796`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:23.109173 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b26893cc-2f2c-4233-84b0-298f0a4ac1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:23.142897 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392b8fa-6fa1-456c-a882-0e490cc1ca81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:23.143513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392b8fa-6fa1-456c-a882-0e490cc1ca81 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:23.143776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392b8fa-6fa1-456c-a882-0e490cc1ca81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:23.144045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392b8fa-6fa1-456c-a882-0e490cc1ca81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:23.187066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1108/2207] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:30:23.198650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26893cc-2f2c-4233-84b0-298f0a4ac1ab None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:23.200187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b26893cc-2f2c-4233-84b0-298f0a4ac1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:23.208202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392b8fa-6fa1-456c-a882-0e490cc1ca81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:23.209679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392b8fa-6fa1-456c-a882-0e490cc1ca81 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:23.212526 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26893cc-2f2c-4233-84b0-298f0a4ac1ab None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:23.216616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd14b6-6c4e-4932-b1ab-99f3567bec4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:23.217247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd14b6-6c4e-4932-b1ab-99f3567bec4e service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:23.217546 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd14b6-6c4e-4932-b1ab-99f3567bec4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:23.217794 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd14b6-6c4e-4932-b1ab-99f3567bec4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:23.219428 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1100/2208] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:23 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:30:23.227693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd14b6-6c4e-4932-b1ab-99f3567bec4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:23.229166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd14b6-6c4e-4932-b1ab-99f3567bec4e service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:23.238382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1109/2209] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:30:23.303231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392b8fa-6fa1-456c-a882-0e490cc1ca81 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:23.312250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392b8fa-6fa1-456c-a882-0e490cc1ca81 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:23.319174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf97c85-814d-48b9-bef8-382b4fbff8ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:23.319835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf97c85-814d-48b9-bef8-382b4fbff8ef service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:23.320111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf97c85-814d-48b9-bef8-382b4fbff8ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:23.320396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf97c85-814d-48b9-bef8-382b4fbff8ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:23.322246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-edf97c85-814d-48b9-bef8-382b4fbff8ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:23.374350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:23.400742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf97c85-814d-48b9-bef8-382b4fbff8ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:23.402557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf97c85-814d-48b9-bef8-382b4fbff8ef service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:23.413868 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1101/2210] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:23 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:30:33.234105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=38 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:33.407960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=8 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:38.431653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd14b6-6c4e-4932-b1ab-99f3567bec4e service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:38.433874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c3dd14b6-6c4e-4932-b1ab-99f3567bec4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:38.624686 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd14b6-6c4e-4932-b1ab-99f3567bec4e service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:38.644936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:38.645708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:38.645984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:38.646260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] PATH_INFO: `/v3/users/e99b1ae919154f03b2a186e444b6d52e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:38.647593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e99b1ae919154f03b2a186e444b6d52e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:38.650179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:38.784931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e99b1ae919154f03b2a186e444b6d52e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:38.805663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e99b1ae919154f03b2a186e444b6d52e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:38.809850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e99b1ae919154f03b2a186e444b6d52e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:38.845447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e99b1ae919154f03b2a186e444b6d52e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:38.878723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e99b1ae919154f03b2a186e444b6d52e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:38.925831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1110/2211] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:30:38 2022] DELETE /identity/v3/users/e99b1ae919154f03b2a186e444b6d52e => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:38.932498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36540-051d-4dc3-8b68-271d79e5baa9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:38.932972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36540-051d-4dc3-8b68-271d79e5baa9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:38.933353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36540-051d-4dc3-8b68-271d79e5baa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:38.993471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-82a36540-051d-4dc3-8b68-271d79e5baa9 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:39.000503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-82a36540-051d-4dc3-8b68-271d79e5baa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:39.147414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1102/2212] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:39.161043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:39.165332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:39.267595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d62b9-14ac-413f-b048-70e45bd811b8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:39.272866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdb7a0-6230-48b5-8d4f-b1f34e6aa505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:39.273847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdb7a0-6230-48b5-8d4f-b1f34e6aa505 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:39.274455 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdb7a0-6230-48b5-8d4f-b1f34e6aa505 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:39.274933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdb7a0-6230-48b5-8d4f-b1f34e6aa505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:39.297075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdb7a0-6230-48b5-8d4f-b1f34e6aa505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:39.298926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdb7a0-6230-48b5-8d4f-b1f34e6aa505 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:39.314869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1111/2213] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:30:39.703527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36540-051d-4dc3-8b68-271d79e5baa9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:39.705155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-82a36540-051d-4dc3-8b68-271d79e5baa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:39.717773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36540-051d-4dc3-8b68-271d79e5baa9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:39.722666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:39.723353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:39.723637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:39.723908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba admin admin] PATH_INFO: `/v3/projects/7db3c3282ab64e6c8fc13ef85a787833` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:39.725423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7db3c3282ab64e6c8fc13ef85a787833)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:39.727322 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:39.774963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7db3c3282ab64e6c8fc13ef85a787833', 'request_id': 'req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:39.783501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7db3c3282ab64e6c8fc13ef85a787833', 'request_id': 'req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:39.790731 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7db3c3282ab64e6c8fc13ef85a787833', 'request_id': 'req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:39.802835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba admin admin] The token cache is being invalidate because project 7db3c3282ab64e6c8fc13ef85a787833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:30:39.803417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:39.804019 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1c6a83ba-89a6-42b4-8d6d-8db87c0098ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:30:39.805902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1103/2214] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:30:39 2022] DELETE /identity/v3/projects/7db3c3282ab64e6c8fc13ef85a787833 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:39.810641 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e27af2-2004-4135-a340-ebde73c58ad7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:39.811097 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e27af2-2004-4135-a340-ebde73c58ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:39.811312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e27af2-2004-4135-a340-ebde73c58ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:39.832025 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-05e27af2-2004-4135-a340-ebde73c58ad7 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:39.833318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-05e27af2-2004-4135-a340-ebde73c58ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:39.853193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:39.896212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1112/2215] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:30:39.904353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508724d7-e526-431f-80d3-0bb32d152279 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:39.904717 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508724d7-e526-431f-80d3-0bb32d152279 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:39.904993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508724d7-e526-431f-80d3-0bb32d152279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:39.924921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-508724d7-e526-431f-80d3-0bb32d152279 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:39.926220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-508724d7-e526-431f-80d3-0bb32d152279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:39.934119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1104/2216] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:39.938362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e27af2-2004-4135-a340-ebde73c58ad7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:39.948609 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e27af2-2004-4135-a340-ebde73c58ad7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:39.953661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfa873-c46a-4bce-a985-be9765bdd24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:39.954282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfa873-c46a-4bce-a985-be9765bdd24b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:39.954555 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfa873-c46a-4bce-a985-be9765bdd24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:39.954900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfa873-c46a-4bce-a985-be9765bdd24b admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:39.955453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfa873-c46a-4bce-a985-be9765bdd24b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:30:39.955764 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfa873-c46a-4bce-a985-be9765bdd24b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:39.957522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfa873-c46a-4bce-a985-be9765bdd24b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:39.970506 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1113/2217] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:30:39.977115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508724d7-e526-431f-80d3-0bb32d152279 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:39.989629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508724d7-e526-431f-80d3-0bb32d152279 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:39.994584 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf386d-6808-4344-af93-b8ed39ce3093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:39.995199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf386d-6808-4344-af93-b8ed39ce3093 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:39.995476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf386d-6808-4344-af93-b8ed39ce3093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:39.995749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf386d-6808-4344-af93-b8ed39ce3093 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:39.996390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf386d-6808-4344-af93-b8ed39ce3093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:39.998341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf386d-6808-4344-af93-b8ed39ce3093 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:40.027392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1105/2218] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:30:39 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:30:40.030872 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfa873-c46a-4bce-a985-be9765bdd24b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:40.041262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfa873-c46a-4bce-a985-be9765bdd24b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:40.116443 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d9859-1707-4242-8912-339687d0cd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:40.117015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d9859-1707-4242-8912-339687d0cd3f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:40.117414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d9859-1707-4242-8912-339687d0cd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:40.117692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d9859-1707-4242-8912-339687d0cd3f admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:40.118366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d9859-1707-4242-8912-339687d0cd3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:40.120545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d9859-1707-4242-8912-339687d0cd3f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:40.144987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1114/2219] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:30:40 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:30:40.149158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf386d-6808-4344-af93-b8ed39ce3093 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:40.160849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf386d-6808-4344-af93-b8ed39ce3093 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:40.165988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4913c-fb33-4f6c-9151-103ef02d36dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:40.166816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4913c-fb33-4f6c-9151-103ef02d36dc admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:40.167269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4913c-fb33-4f6c-9151-103ef02d36dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:40.167730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4913c-fb33-4f6c-9151-103ef02d36dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:40.168603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4913c-fb33-4f6c-9151-103ef02d36dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:40.174595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4913c-fb33-4f6c-9151-103ef02d36dc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:40.185643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1106/2220] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:30:40.190046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d9859-1707-4242-8912-339687d0cd3f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:40.200540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d9859-1707-4242-8912-339687d0cd3f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:40.205487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefe585-10ea-4b27-babf-5ba7803fbfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:40.206217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefe585-10ea-4b27-babf-5ba7803fbfd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:40.206559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefe585-10ea-4b27-babf-5ba7803fbfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:40.206899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefe585-10ea-4b27-babf-5ba7803fbfd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:40.207614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefe585-10ea-4b27-babf-5ba7803fbfd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:40.209350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefe585-10ea-4b27-babf-5ba7803fbfd1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:40.220692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1115/2221] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:30:40.225307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4913c-fb33-4f6c-9151-103ef02d36dc admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:40.235807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4913c-fb33-4f6c-9151-103ef02d36dc admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:40.240587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb3e55-5f92-4127-adc2-13ed6814d5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:40.241361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb3e55-5f92-4127-adc2-13ed6814d5e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:40.241611 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb3e55-5f92-4127-adc2-13ed6814d5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:40.241881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb3e55-5f92-4127-adc2-13ed6814d5e5 admin admin] PATH_INFO: `/v3/projects/752e65efe679458fab6b8dbc3a9de82f/users/ff241cbd3f7c47328ac5ef2ed4c66e18/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:40.257987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb3e55-5f92-4127-adc2-13ed6814d5e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=752e65efe679458fab6b8dbc3a9de82f, user_id=ff241cbd3f7c47328ac5ef2ed4c66e18, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:40.260614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb3e55-5f92-4127-adc2-13ed6814d5e5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:40.275409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1107/2222] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:30:40 2022] PUT /identity/v3/projects/752e65efe679458fab6b8dbc3a9de82f/users/ff241cbd3f7c47328ac5ef2ed4c66e18/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:40.280856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea74ae4-97ab-4105-9227-7e06e2bdb7b2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:40.281162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea74ae4-97ab-4105-9227-7e06e2bdb7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:40.281434 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea74ae4-97ab-4105-9227-7e06e2bdb7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:40.302103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bea74ae4-97ab-4105-9227-7e06e2bdb7b2 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:40.303220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bea74ae4-97ab-4105-9227-7e06e2bdb7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:40.373962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1116/2223] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:40.383914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb3e55-5f92-4127-adc2-13ed6814d5e5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:40.385695 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0ceb3e55-5f92-4127-adc2-13ed6814d5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:40.408609 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:40.448150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb3e55-5f92-4127-adc2-13ed6814d5e5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:40.452225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af137b-ae9d-4d64-a1c7-c5e6153de2b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:40.452838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af137b-ae9d-4d64-a1c7-c5e6153de2b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:40.453185 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af137b-ae9d-4d64-a1c7-c5e6153de2b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:40.453543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af137b-ae9d-4d64-a1c7-c5e6153de2b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:40.463623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af137b-ae9d-4d64-a1c7-c5e6153de2b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:40.465217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af137b-ae9d-4d64-a1c7-c5e6153de2b3 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:40.475338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1108/2224] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:30:44.938375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e0309-3279-4ea0-8f39-1f0da9ee1af1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:44.938841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e0309-3279-4ea0-8f39-1f0da9ee1af1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:44.938921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e0309-3279-4ea0-8f39-1f0da9ee1af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:44.982753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-106e0309-3279-4ea0-8f39-1f0da9ee1af1 None None] MFA Rules not processed for user `ff241cbd3f7c47328ac5ef2ed4c66e18`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:44.984323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-106e0309-3279-4ea0-8f39-1f0da9ee1af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:45.002082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:45.060893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1117/2225] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:45.065565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af137b-ae9d-4d64-a1c7-c5e6153de2b3 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:45.075569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af137b-ae9d-4d64-a1c7-c5e6153de2b3 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:45.080279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15819b3-325a-4cc1-a758-d69826176de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:45.081052 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15819b3-325a-4cc1-a758-d69826176de7 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:45.081333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15819b3-325a-4cc1-a758-d69826176de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:45.081603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15819b3-325a-4cc1-a758-d69826176de7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:45.085959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15819b3-325a-4cc1-a758-d69826176de7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:45.085959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15819b3-325a-4cc1-a758-d69826176de7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:45.114024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1109/2226] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:30:45 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:30:45.118204 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e0309-3279-4ea0-8f39-1f0da9ee1af1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:45.129509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e0309-3279-4ea0-8f39-1f0da9ee1af1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:45.186676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cb74b-1261-4424-9854-ea5e883c6b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:45.187297 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb74b-1261-4424-9854-ea5e883c6b68 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:45.187576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb74b-1261-4424-9854-ea5e883c6b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:45.187844 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb74b-1261-4424-9854-ea5e883c6b68 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:45.188492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cb74b-1261-4424-9854-ea5e883c6b68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:45.190788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cb74b-1261-4424-9854-ea5e883c6b68 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:45.215304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1118/2227] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:30:45 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:30:45.219411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15819b3-325a-4cc1-a758-d69826176de7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:45.230952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15819b3-325a-4cc1-a758-d69826176de7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:45.235675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a540e-c189-4d3f-a0ef-2da39eb854ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:45.236337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a540e-c189-4d3f-a0ef-2da39eb854ea admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:45.236618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a540e-c189-4d3f-a0ef-2da39eb854ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:45.236982 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a540e-c189-4d3f-a0ef-2da39eb854ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:45.237728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a540e-c189-4d3f-a0ef-2da39eb854ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:45.239345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a540e-c189-4d3f-a0ef-2da39eb854ea admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:45.251016 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1110/2228] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:30:45.255559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cb74b-1261-4424-9854-ea5e883c6b68 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:45.266209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cb74b-1261-4424-9854-ea5e883c6b68 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:45.271669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223d8dc-b46b-4430-a50e-b7a24c52961a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:45.272461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223d8dc-b46b-4430-a50e-b7a24c52961a admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:45.272745 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223d8dc-b46b-4430-a50e-b7a24c52961a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:45.273015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223d8dc-b46b-4430-a50e-b7a24c52961a admin admin] PATH_INFO: `/v3/projects/7a5abdc0c6ab45338c7c41910f231a30/users/b1532b271e4d477eabcfaf602b684bd3/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:45.289754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223d8dc-b46b-4430-a50e-b7a24c52961a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5abdc0c6ab45338c7c41910f231a30, user_id=b1532b271e4d477eabcfaf602b684bd3, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:45.292396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223d8dc-b46b-4430-a50e-b7a24c52961a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:45.311167 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1119/2229] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:30:45 2022] PUT /identity/v3/projects/7a5abdc0c6ab45338c7c41910f231a30/users/b1532b271e4d477eabcfaf602b684bd3/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:30:45.312351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a540e-c189-4d3f-a0ef-2da39eb854ea admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:45.323395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a540e-c189-4d3f-a0ef-2da39eb854ea admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:45.435303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d01ad-41a5-456f-83a2-f3bb6602a817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:45.436024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d01ad-41a5-456f-83a2-f3bb6602a817 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:45.436369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d01ad-41a5-456f-83a2-f3bb6602a817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:45.436942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d01ad-41a5-456f-83a2-f3bb6602a817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:45.437706 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d01ad-41a5-456f-83a2-f3bb6602a817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:45.439315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d01ad-41a5-456f-83a2-f3bb6602a817 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:45.450277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1111/2230] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:30:45.461503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223d8dc-b46b-4430-a50e-b7a24c52961a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:45.479886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223d8dc-b46b-4430-a50e-b7a24c52961a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:45.490950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63ae25-4cf4-45a4-80de-ecf9f1a2a790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:45.491645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63ae25-4cf4-45a4-80de-ecf9f1a2a790 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:45.491916 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63ae25-4cf4-45a4-80de-ecf9f1a2a790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:45.492349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63ae25-4cf4-45a4-80de-ecf9f1a2a790 admin admin] PATH_INFO: `/v3/projects/7a5abdc0c6ab45338c7c41910f231a30/users/b1532b271e4d477eabcfaf602b684bd3/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:45.494371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63ae25-4cf4-45a4-80de-ecf9f1a2a790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5abdc0c6ab45338c7c41910f231a30, user_id=b1532b271e4d477eabcfaf602b684bd3, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:45.497044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63ae25-4cf4-45a4-80de-ecf9f1a2a790 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:45.510935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1120/2231] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:30:45 2022] PUT /identity/v3/projects/7a5abdc0c6ab45338c7c41910f231a30/users/b1532b271e4d477eabcfaf602b684bd3/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:30:50.662779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c35e73-7810-4f2f-982d-838cc8296956 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:50.663694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c35e73-7810-4f2f-982d-838cc8296956 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:50.664225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c35e73-7810-4f2f-982d-838cc8296956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:50.700261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-22c35e73-7810-4f2f-982d-838cc8296956 None None] MFA Rules not processed for user `b1532b271e4d477eabcfaf602b684bd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:30:50.702744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-22c35e73-7810-4f2f-982d-838cc8296956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:50.840571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1112/2232] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:30:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:30:50.848327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63ae25-4cf4-45a4-80de-ecf9f1a2a790 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:30:50.851185 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2d63ae25-4cf4-45a4-80de-ecf9f1a2a790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:30:50.953809 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63ae25-4cf4-45a4-80de-ecf9f1a2a790 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:30:50.960042 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e572e-03fe-4cd6-99f3-a4df52fd1ffd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:30:50.960856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e572e-03fe-4cd6-99f3-a4df52fd1ffd service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:30:50.961170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e572e-03fe-4cd6-99f3-a4df52fd1ffd service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:30:50.961445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e572e-03fe-4cd6-99f3-a4df52fd1ffd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:30:50.969033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:51.046152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e572e-03fe-4cd6-99f3-a4df52fd1ffd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:30:51.048988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e572e-03fe-4cd6-99f3-a4df52fd1ffd service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:30:51.074253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1121/2233] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:30:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:00.827335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:01.063393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=24 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:03.890443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c35e73-7810-4f2f-982d-838cc8296956 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:03.891964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-22c35e73-7810-4f2f-982d-838cc8296956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:03.963074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c35e73-7810-4f2f-982d-838cc8296956 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:03.968059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:03.968717 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:03.968999 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:03.969306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] PATH_INFO: `/v3/users/85284acd929d4042ba6c42454aa68e70` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:03.970539 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] RBAC: Authorizing `identity:delete_user(user_id=85284acd929d4042ba6c42454aa68e70)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:03.972500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:04.020424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85284acd929d4042ba6c42454aa68e70'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.031890 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85284acd929d4042ba6c42454aa68e70'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.032747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85284acd929d4042ba6c42454aa68e70'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.040793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85284acd929d4042ba6c42454aa68e70'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.053644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85284acd929d4042ba6c42454aa68e70'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.059837 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1113/2234] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:31:03 2022] DELETE /identity/v3/users/85284acd929d4042ba6c42454aa68e70 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:04.064561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa3284-faeb-4c3a-9b24-8fd364f811c6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:04.065108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa3284-faeb-4c3a-9b24-8fd364f811c6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:04.065108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa3284-faeb-4c3a-9b24-8fd364f811c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:04.085141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-edfa3284-faeb-4c3a-9b24-8fd364f811c6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:04.086518 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-edfa3284-faeb-4c3a-9b24-8fd364f811c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:04.151112 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1122/2235] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:04.161638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:04.163517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:04.227505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2a4f6-4cb3-4927-82fb-59753dc27faa admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:04.233174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32c033-1ace-41da-a85e-4aeffaae873f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:04.234036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32c033-1ace-41da-a85e-4aeffaae873f service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:04.234371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32c033-1ace-41da-a85e-4aeffaae873f service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:04.234711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32c033-1ace-41da-a85e-4aeffaae873f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:04.245598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32c033-1ace-41da-a85e-4aeffaae873f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:04.247180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32c033-1ace-41da-a85e-4aeffaae873f service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:04.256903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1114/2236] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:04.569924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa3284-faeb-4c3a-9b24-8fd364f811c6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:04.571344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-edfa3284-faeb-4c3a-9b24-8fd364f811c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:04.587785 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa3284-faeb-4c3a-9b24-8fd364f811c6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:04.593096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:04.593917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:04.594192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:04.594483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] PATH_INFO: `/v3/projects/2fdd592ec7c042b28c573479dc08d3b7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:04.595811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fdd592ec7c042b28c573479dc08d3b7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:04.597761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:04.649343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fdd592ec7c042b28c573479dc08d3b7', 'request_id': 'req-0e2c4a8a-7d67-4332-a3a3-d81a61902210'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.662847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fdd592ec7c042b28c573479dc08d3b7', 'request_id': 'req-0e2c4a8a-7d67-4332-a3a3-d81a61902210'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.671006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fdd592ec7c042b28c573479dc08d3b7', 'request_id': 'req-0e2c4a8a-7d67-4332-a3a3-d81a61902210'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.684536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] The token cache is being invalidate because project 2fdd592ec7c042b28c573479dc08d3b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:31:04.684960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.685735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.687586 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1123/2237] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:31:04 2022] DELETE /identity/v3/projects/2fdd592ec7c042b28c573479dc08d3b7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:04.691854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32c033-1ace-41da-a85e-4aeffaae873f service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:04.702295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32c033-1ace-41da-a85e-4aeffaae873f service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:04.726781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:04.775916 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2dfcf-043a-4aeb-b63b-b7071f317ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:04.777204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2dfcf-043a-4aeb-b63b-b7071f317ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:04.777581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2dfcf-043a-4aeb-b63b-b7071f317ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:04.778278 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2dfcf-043a-4aeb-b63b-b7071f317ca5 admin admin] PATH_INFO: `/v3/users/17e15c47755a4247a57901d71171f5cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:04.779787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2dfcf-043a-4aeb-b63b-b7071f317ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e15c47755a4247a57901d71171f5cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:04.782362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2dfcf-043a-4aeb-b63b-b7071f317ca5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:04.861671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bed2dfcf-043a-4aeb-b63b-b7071f317ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e15c47755a4247a57901d71171f5cc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.867698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bed2dfcf-043a-4aeb-b63b-b7071f317ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e15c47755a4247a57901d71171f5cc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.868316 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bed2dfcf-043a-4aeb-b63b-b7071f317ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e15c47755a4247a57901d71171f5cc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.876353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bed2dfcf-043a-4aeb-b63b-b7071f317ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e15c47755a4247a57901d71171f5cc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.890899 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bed2dfcf-043a-4aeb-b63b-b7071f317ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e15c47755a4247a57901d71171f5cc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:04.902412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1115/2238] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:31:04 2022] DELETE /identity/v3/users/17e15c47755a4247a57901d71171f5cc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:31:05.193808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:05.195511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:05.214251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4a8a-7d67-4332-a3a3-d81a61902210 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:05.277421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:05.278061 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:05.278343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:05.278611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b admin admin] PATH_INFO: `/v3/projects/b955c4e4289548feaeaecc0081760305` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:05.279724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b955c4e4289548feaeaecc0081760305)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:05.281952 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:05.314462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:05.334073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b955c4e4289548feaeaecc0081760305', 'request_id': 'req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:05.342674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b955c4e4289548feaeaecc0081760305', 'request_id': 'req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:05.348572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b955c4e4289548feaeaecc0081760305', 'request_id': 'req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:05.360144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b admin admin] The token cache is being invalidate because project b955c4e4289548feaeaecc0081760305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:31:05.360425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:05.361103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7c1ec1a0-54c2-4d19-9bcc-d85bdcf8141b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:05.362648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1124/2239] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:31:05 2022] DELETE /identity/v3/projects/b955c4e4289548feaeaecc0081760305 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:31:05.368444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa8dda-c890-4a9e-b728-4c8a5c040b9d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:05.368797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa8dda-c890-4a9e-b728-4c8a5c040b9d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:05.369307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa8dda-c890-4a9e-b728-4c8a5c040b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:05.392271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9daa8dda-c890-4a9e-b728-4c8a5c040b9d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:05.393530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9daa8dda-c890-4a9e-b728-4c8a5c040b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:05.472030 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1116/2240] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:05.481905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef11d0-69cf-4ed7-b907-4201c55e6ce8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:05.482077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef11d0-69cf-4ed7-b907-4201c55e6ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:05.482323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef11d0-69cf-4ed7-b907-4201c55e6ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:05.505024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-62ef11d0-69cf-4ed7-b907-4201c55e6ce8 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:05.506273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-62ef11d0-69cf-4ed7-b907-4201c55e6ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:05.514918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1125/2241] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:05.519376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa8dda-c890-4a9e-b728-4c8a5c040b9d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:05.529406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa8dda-c890-4a9e-b728-4c8a5c040b9d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:05.535101 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9272f-4458-48cf-b36e-257c168b1900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:05.535819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9272f-4458-48cf-b36e-257c168b1900 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:05.536089 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9272f-4458-48cf-b36e-257c168b1900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:05.536358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9272f-4458-48cf-b36e-257c168b1900 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:05.537021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9272f-4458-48cf-b36e-257c168b1900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:31:05.537368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9272f-4458-48cf-b36e-257c168b1900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:05.538970 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9272f-4458-48cf-b36e-257c168b1900 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:05.551123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1117/2242] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:31:05.555596 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef11d0-69cf-4ed7-b907-4201c55e6ce8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:05.565107 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef11d0-69cf-4ed7-b907-4201c55e6ce8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:05.570430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0c6e7-b93a-47b6-a9fc-3c9d4cc4561b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:05.571040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0c6e7-b93a-47b6-a9fc-3c9d4cc4561b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:05.571312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0c6e7-b93a-47b6-a9fc-3c9d4cc4561b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:05.571576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0c6e7-b93a-47b6-a9fc-3c9d4cc4561b admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:05.572214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0c6e7-b93a-47b6-a9fc-3c9d4cc4561b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:05.574146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0c6e7-b93a-47b6-a9fc-3c9d4cc4561b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:05.603131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1126/2243] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:31:05 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:31:05.607462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9272f-4458-48cf-b36e-257c168b1900 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:05.617398 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9272f-4458-48cf-b36e-257c168b1900 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:05.644000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:05.677007 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dcb31-253f-4905-8603-6a31f1c8fbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:05.677623 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dcb31-253f-4905-8603-6a31f1c8fbf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:05.677893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dcb31-253f-4905-8603-6a31f1c8fbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:05.678159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dcb31-253f-4905-8603-6a31f1c8fbf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:05.678858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3dcb31-253f-4905-8603-6a31f1c8fbf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:05.680779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3dcb31-253f-4905-8603-6a31f1c8fbf8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:05.704185 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1118/2244] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:31:05 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:31:05.708459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0c6e7-b93a-47b6-a9fc-3c9d4cc4561b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:05.721194 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0c6e7-b93a-47b6-a9fc-3c9d4cc4561b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:05.727432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39847c-793a-4a59-8eff-a56a6dcb0610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:05.728055 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39847c-793a-4a59-8eff-a56a6dcb0610 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:05.728331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39847c-793a-4a59-8eff-a56a6dcb0610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:05.728600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39847c-793a-4a59-8eff-a56a6dcb0610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:05.729286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39847c-793a-4a59-8eff-a56a6dcb0610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:05.730919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39847c-793a-4a59-8eff-a56a6dcb0610 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:05.743476 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1127/2245] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:31:05.748079 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dcb31-253f-4905-8603-6a31f1c8fbf8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:05.760534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dcb31-253f-4905-8603-6a31f1c8fbf8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:05.766576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac847c2f-40db-4578-9fe1-8da47d037228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:05.767281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac847c2f-40db-4578-9fe1-8da47d037228 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:05.767555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac847c2f-40db-4578-9fe1-8da47d037228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:05.767824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac847c2f-40db-4578-9fe1-8da47d037228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:05.768475 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac847c2f-40db-4578-9fe1-8da47d037228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:05.770276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac847c2f-40db-4578-9fe1-8da47d037228 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:05.782045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1119/2246] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:31:05.786664 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39847c-793a-4a59-8eff-a56a6dcb0610 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:05.813381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39847c-793a-4a59-8eff-a56a6dcb0610 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:05.813381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0ba6c-5440-4f95-8493-a1e714881a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:05.813381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0ba6c-5440-4f95-8493-a1e714881a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:05.813381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0ba6c-5440-4f95-8493-a1e714881a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:05.813767 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0ba6c-5440-4f95-8493-a1e714881a1f admin admin] PATH_INFO: `/v3/projects/bb360e404c2542cdb9e23afe3c616d7a/users/b17e203d89ff4ae0be152ac496f5b507/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:05.823811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0ba6c-5440-4f95-8493-a1e714881a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb360e404c2542cdb9e23afe3c616d7a, user_id=b17e203d89ff4ae0be152ac496f5b507, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:05.826557 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0ba6c-5440-4f95-8493-a1e714881a1f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:05.840348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1128/2247] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:31:05 2022] PUT /identity/v3/projects/bb360e404c2542cdb9e23afe3c616d7a/users/b17e203d89ff4ae0be152ac496f5b507/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:05.851512 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428948f0-83d0-4249-9833-18f0723ceed5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:05.851782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428948f0-83d0-4249-9833-18f0723ceed5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:05.852051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428948f0-83d0-4249-9833-18f0723ceed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:05.883738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-428948f0-83d0-4249-9833-18f0723ceed5 None None] MFA Rules not processed for user `b17e203d89ff4ae0be152ac496f5b507`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:05.885234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-428948f0-83d0-4249-9833-18f0723ceed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:05.975520 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1120/2248] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:05.979870 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0ba6c-5440-4f95-8493-a1e714881a1f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:06.093784 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0ba6c-5440-4f95-8493-a1e714881a1f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:06.098760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341afe1-36d1-4497-b0f9-de9ce134911c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:06.099691 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341afe1-36d1-4497-b0f9-de9ce134911c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:06.099964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341afe1-36d1-4497-b0f9-de9ce134911c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:06.100229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341afe1-36d1-4497-b0f9-de9ce134911c admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:06.100859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341afe1-36d1-4497-b0f9-de9ce134911c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:06.102821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341afe1-36d1-4497-b0f9-de9ce134911c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:06.131625 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1129/2249] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:31:05 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:31:06.135793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428948f0-83d0-4249-9833-18f0723ceed5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:06.146606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428948f0-83d0-4249-9833-18f0723ceed5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:06.205893 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b077bd-0b2f-413b-86db-608d13eb6631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:06.206539 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b077bd-0b2f-413b-86db-608d13eb6631 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:06.206808 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b077bd-0b2f-413b-86db-608d13eb6631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:06.207075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b077bd-0b2f-413b-86db-608d13eb6631 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:06.207734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b077bd-0b2f-413b-86db-608d13eb6631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:06.209684 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b077bd-0b2f-413b-86db-608d13eb6631 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:06.232347 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1121/2250] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:31:06 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:31:06.236711 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341afe1-36d1-4497-b0f9-de9ce134911c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:06.247486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341afe1-36d1-4497-b0f9-de9ce134911c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:06.252343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15df2d5-11eb-4e57-a24e-8cb6466aaa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:06.252930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15df2d5-11eb-4e57-a24e-8cb6466aaa96 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:06.253240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15df2d5-11eb-4e57-a24e-8cb6466aaa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:06.253507 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15df2d5-11eb-4e57-a24e-8cb6466aaa96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:06.254151 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15df2d5-11eb-4e57-a24e-8cb6466aaa96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:06.255903 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15df2d5-11eb-4e57-a24e-8cb6466aaa96 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:06.266328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1130/2251] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:31:06.270594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b077bd-0b2f-413b-86db-608d13eb6631 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:06.281338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b077bd-0b2f-413b-86db-608d13eb6631 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:06.287512 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862f8af-4f89-4fa4-98be-3fe5162699ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:06.288270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862f8af-4f89-4fa4-98be-3fe5162699ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:06.288526 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862f8af-4f89-4fa4-98be-3fe5162699ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:06.288826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862f8af-4f89-4fa4-98be-3fe5162699ab admin admin] PATH_INFO: `/v3/projects/90f1a3cd8d674fb3ba0a828b6b646464/users/e2b53134b0184702803b85c102d3783f/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:06.304203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862f8af-4f89-4fa4-98be-3fe5162699ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f1a3cd8d674fb3ba0a828b6b646464, user_id=e2b53134b0184702803b85c102d3783f, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:06.306930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862f8af-4f89-4fa4-98be-3fe5162699ab admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:06.321902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1122/2252] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:31:06 2022] PUT /identity/v3/projects/90f1a3cd8d674fb3ba0a828b6b646464/users/e2b53134b0184702803b85c102d3783f/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:06.326172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15df2d5-11eb-4e57-a24e-8cb6466aaa96 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:06.336701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15df2d5-11eb-4e57-a24e-8cb6466aaa96 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:06.357466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:06.398898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f5370-7e69-482d-81f6-7ef997bc30bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:06.399522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f5370-7e69-482d-81f6-7ef997bc30bd admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:06.399850 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f5370-7e69-482d-81f6-7ef997bc30bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:06.400128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f5370-7e69-482d-81f6-7ef997bc30bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:06.400957 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f5370-7e69-482d-81f6-7ef997bc30bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:06.402693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f5370-7e69-482d-81f6-7ef997bc30bd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:06.414520 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1131/2253] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:31:06.419159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862f8af-4f89-4fa4-98be-3fe5162699ab admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:06.425027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:06.431920 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862f8af-4f89-4fa4-98be-3fe5162699ab admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:06.436932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c88b5d-d0d5-4ed5-ab0a-14d6afe3cd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:06.437630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c88b5d-d0d5-4ed5-ab0a-14d6afe3cd28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:06.437935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c88b5d-d0d5-4ed5-ab0a-14d6afe3cd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:06.438357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c88b5d-d0d5-4ed5-ab0a-14d6afe3cd28 admin admin] PATH_INFO: `/v3/projects/90f1a3cd8d674fb3ba0a828b6b646464/users/e2b53134b0184702803b85c102d3783f/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:06.440286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c88b5d-d0d5-4ed5-ab0a-14d6afe3cd28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f1a3cd8d674fb3ba0a828b6b646464, user_id=e2b53134b0184702803b85c102d3783f, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:06.443084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c88b5d-d0d5-4ed5-ab0a-14d6afe3cd28 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:06.453794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1123/2254] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:31:06 2022] PUT /identity/v3/projects/90f1a3cd8d674fb3ba0a828b6b646464/users/e2b53134b0184702803b85c102d3783f/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:31:06.464465 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1fa00-39a4-45b8-8761-01e7d4efe584 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:06.464776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1fa00-39a4-45b8-8761-01e7d4efe584 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:06.465052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1fa00-39a4-45b8-8761-01e7d4efe584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:06.495989 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bda1fa00-39a4-45b8-8761-01e7d4efe584 None None] MFA Rules not processed for user `e2b53134b0184702803b85c102d3783f`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:06.497242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bda1fa00-39a4-45b8-8761-01e7d4efe584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:06.570969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1132/2255] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:06.582430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c88b5d-d0d5-4ed5-ab0a-14d6afe3cd28 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:06.584434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-04c88b5d-d0d5-4ed5-ab0a-14d6afe3cd28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:06.647598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c88b5d-d0d5-4ed5-ab0a-14d6afe3cd28 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:06.651529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56051d5-c37a-491f-8fc6-9b4a90c4afaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:06.652187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56051d5-c37a-491f-8fc6-9b4a90c4afaa service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:06.652449 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56051d5-c37a-491f-8fc6-9b4a90c4afaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:06.652717 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56051d5-c37a-491f-8fc6-9b4a90c4afaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:06.662236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56051d5-c37a-491f-8fc6-9b4a90c4afaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:06.663690 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56051d5-c37a-491f-8fc6-9b4a90c4afaa service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:06.673486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1124/2256] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:07.388638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1fa00-39a4-45b8-8761-01e7d4efe584 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:07.457344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1fa00-39a4-45b8-8761-01e7d4efe584 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:07.462997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:07.463837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:07.464841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:07.464841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] PATH_INFO: `/v3/users/b17e203d89ff4ae0be152ac496f5b507` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:07.466183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b17e203d89ff4ae0be152ac496f5b507)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:07.468101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:07.527409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b17e203d89ff4ae0be152ac496f5b507'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:07.529189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:07.538128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b17e203d89ff4ae0be152ac496f5b507'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:07.538922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b17e203d89ff4ae0be152ac496f5b507'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:07.548585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b17e203d89ff4ae0be152ac496f5b507'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:07.573388 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b17e203d89ff4ae0be152ac496f5b507'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:07.586679 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1133/2257] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:31:07 2022] DELETE /identity/v3/users/b17e203d89ff4ae0be152ac496f5b507 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:07.590755 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d81ec-c5a7-476a-978c-dd2befd5c367 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:07.591330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d81ec-c5a7-476a-978c-dd2befd5c367 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:07.591581 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d81ec-c5a7-476a-978c-dd2befd5c367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:07.623798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c95d81ec-c5a7-476a-978c-dd2befd5c367 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:07.625148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c95d81ec-c5a7-476a-978c-dd2befd5c367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:07.687803 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1125/2258] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:31:07.698301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:07.700127 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:07.765420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21be6d-2120-41b5-b679-ed2a03ebd5f3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:07.769439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483df63-b04e-4a6d-b437-f7d3df39d33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:07.770088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483df63-b04e-4a6d-b437-f7d3df39d33b service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:07.770365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483df63-b04e-4a6d-b437-f7d3df39d33b service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:07.770636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483df63-b04e-4a6d-b437-f7d3df39d33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:07.779902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483df63-b04e-4a6d-b437-f7d3df39d33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:07.781361 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483df63-b04e-4a6d-b437-f7d3df39d33b service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:07.796129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1134/2259] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:08.085947 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d81ec-c5a7-476a-978c-dd2befd5c367 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:08.087385 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c95d81ec-c5a7-476a-978c-dd2befd5c367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:08.113045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d81ec-c5a7-476a-978c-dd2befd5c367 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:08.118284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:08.119027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:08.119259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:08.119540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] PATH_INFO: `/v3/projects/bb360e404c2542cdb9e23afe3c616d7a` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:08.120681 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb360e404c2542cdb9e23afe3c616d7a)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:08.122670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:08.175626 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb360e404c2542cdb9e23afe3c616d7a', 'request_id': 'req-89c012e8-577a-46d8-b600-72d2027a85cf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.184784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb360e404c2542cdb9e23afe3c616d7a', 'request_id': 'req-89c012e8-577a-46d8-b600-72d2027a85cf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.191451 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb360e404c2542cdb9e23afe3c616d7a', 'request_id': 'req-89c012e8-577a-46d8-b600-72d2027a85cf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.205942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] The token cache is being invalidate because project bb360e404c2542cdb9e23afe3c616d7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:31:08.205942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.205942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.206337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1126/2260] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:31:08 2022] DELETE /identity/v3/projects/bb360e404c2542cdb9e23afe3c616d7a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:08.210980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483df63-b04e-4a6d-b437-f7d3df39d33b service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:08.221711 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483df63-b04e-4a6d-b437-f7d3df39d33b service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:08.279787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6a21f-626e-482d-9f32-b7453e8985ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:08.280418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6a21f-626e-482d-9f32-b7453e8985ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:08.280684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6a21f-626e-482d-9f32-b7453e8985ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:08.281040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6a21f-626e-482d-9f32-b7453e8985ee admin admin] PATH_INFO: `/v3/users/e2b53134b0184702803b85c102d3783f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:08.282215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6a21f-626e-482d-9f32-b7453e8985ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2b53134b0184702803b85c102d3783f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:08.284133 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6a21f-626e-482d-9f32-b7453e8985ee admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:08.297269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:08.299465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:08.318069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c012e8-577a-46d8-b600-72d2027a85cf admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:08.324734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:08.325494 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:08.325809 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:08.326100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] PATH_INFO: `/v3/users/923d21322c084edcb323f7b431d0f17d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:08.327446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=923d21322c084edcb323f7b431d0f17d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:08.329756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:08.341914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:08.342807 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2a6a21f-626e-482d-9f32-b7453e8985ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2b53134b0184702803b85c102d3783f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.352338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2a6a21f-626e-482d-9f32-b7453e8985ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2b53134b0184702803b85c102d3783f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.353252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2a6a21f-626e-482d-9f32-b7453e8985ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2b53134b0184702803b85c102d3783f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.363425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2a6a21f-626e-482d-9f32-b7453e8985ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2b53134b0184702803b85c102d3783f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.369362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:08.377865 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2a6a21f-626e-482d-9f32-b7453e8985ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2b53134b0184702803b85c102d3783f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.384332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '923d21322c084edcb323f7b431d0f17d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.385516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1135/2261] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:31:08 2022] DELETE /identity/v3/users/e2b53134b0184702803b85c102d3783f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:08.394999 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '923d21322c084edcb323f7b431d0f17d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.395666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '923d21322c084edcb323f7b431d0f17d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.404237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '923d21322c084edcb323f7b431d0f17d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.420122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '923d21322c084edcb323f7b431d0f17d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.427271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1127/2262] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:31:08 2022] DELETE /identity/v3/users/923d21322c084edcb323f7b431d0f17d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:31:08.432118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032ca48-5bd2-4dcc-a236-f7cc203b2a02 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:08.432670 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032ca48-5bd2-4dcc-a236-f7cc203b2a02 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:08.432670 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032ca48-5bd2-4dcc-a236-f7cc203b2a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:08.453164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8032ca48-5bd2-4dcc-a236-f7cc203b2a02 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:08.454414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8032ca48-5bd2-4dcc-a236-f7cc203b2a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:08.521128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1136/2263] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:08.531291 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:08.532750 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:08.611071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c5e57-a655-4b83-85f2-31a23d4a3d9b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:08.615057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee9e87-159f-4571-b879-dd1aa0d0b10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:08.615711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9e87-159f-4571-b879-dd1aa0d0b10b service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:08.615983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9e87-159f-4571-b879-dd1aa0d0b10b service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:08.616248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9e87-159f-4571-b879-dd1aa0d0b10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:08.630854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee9e87-159f-4571-b879-dd1aa0d0b10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:08.632417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee9e87-159f-4571-b879-dd1aa0d0b10b service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:08.644351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1128/2264] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:08.757880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032ca48-5bd2-4dcc-a236-f7cc203b2a02 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:08.759344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8032ca48-5bd2-4dcc-a236-f7cc203b2a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:08.771624 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032ca48-5bd2-4dcc-a236-f7cc203b2a02 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:08.777026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13710925-c718-4295-aceb-58f5f359e489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:08.777725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13710925-c718-4295-aceb-58f5f359e489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:08.778000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13710925-c718-4295-aceb-58f5f359e489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:08.778269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13710925-c718-4295-aceb-58f5f359e489 admin admin] PATH_INFO: `/v3/projects/90f1a3cd8d674fb3ba0a828b6b646464` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:08.779578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13710925-c718-4295-aceb-58f5f359e489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f1a3cd8d674fb3ba0a828b6b646464)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:08.785197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13710925-c718-4295-aceb-58f5f359e489 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:08.843004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-13710925-c718-4295-aceb-58f5f359e489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f1a3cd8d674fb3ba0a828b6b646464', 'request_id': 'req-13710925-c718-4295-aceb-58f5f359e489'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.853826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-13710925-c718-4295-aceb-58f5f359e489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f1a3cd8d674fb3ba0a828b6b646464', 'request_id': 'req-13710925-c718-4295-aceb-58f5f359e489'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.869281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-13710925-c718-4295-aceb-58f5f359e489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f1a3cd8d674fb3ba0a828b6b646464', 'request_id': 'req-13710925-c718-4295-aceb-58f5f359e489'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.883071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-13710925-c718-4295-aceb-58f5f359e489 admin admin] The token cache is being invalidate because project 90f1a3cd8d674fb3ba0a828b6b646464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:31:08.883396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-13710925-c718-4295-aceb-58f5f359e489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.884054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-13710925-c718-4295-aceb-58f5f359e489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:08.885793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1137/2265] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:31:08 2022] DELETE /identity/v3/projects/90f1a3cd8d674fb3ba0a828b6b646464 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:08.890735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd73d9-3289-4939-8fa4-362771a68fb2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:08.891093 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd73d9-3289-4939-8fa4-362771a68fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:08.891375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd73d9-3289-4939-8fa4-362771a68fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:08.914036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-26dd73d9-3289-4939-8fa4-362771a68fb2 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:08.915387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-26dd73d9-3289-4939-8fa4-362771a68fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:08.998381 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1129/2266] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:31:09.007493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a825f-50c5-4b6f-bc57-fa569cc10c88 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.007775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a825f-50c5-4b6f-bc57-fa569cc10c88 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.008049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a825f-50c5-4b6f-bc57-fa569cc10c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.028381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c52a825f-50c5-4b6f-bc57-fa569cc10c88 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:09.029616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c52a825f-50c5-4b6f-bc57-fa569cc10c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:09.045453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1138/2267] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:09.049948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd73d9-3289-4939-8fa4-362771a68fb2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.062599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd73d9-3289-4939-8fa4-362771a68fb2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.067417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64221eb-43e6-4fe6-9874-8cf7876ec069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.068107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64221eb-43e6-4fe6-9874-8cf7876ec069 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.068454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64221eb-43e6-4fe6-9874-8cf7876ec069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.068795 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64221eb-43e6-4fe6-9874-8cf7876ec069 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.069556 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64221eb-43e6-4fe6-9874-8cf7876ec069 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:31:09.069954 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64221eb-43e6-4fe6-9874-8cf7876ec069 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.071800 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64221eb-43e6-4fe6-9874-8cf7876ec069 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.084778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1130/2268] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:31:09.089177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a825f-50c5-4b6f-bc57-fa569cc10c88 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.100087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a825f-50c5-4b6f-bc57-fa569cc10c88 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.100869 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64221eb-43e6-4fe6-9874-8cf7876ec069 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.103169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b64221eb-43e6-4fe6-9874-8cf7876ec069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:09.104859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88883da-d9b8-4a60-95f4-cead0f28c7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.105646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88883da-d9b8-4a60-95f4-cead0f28c7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.105784 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88883da-d9b8-4a60-95f4-cead0f28c7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.106032 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88883da-d9b8-4a60-95f4-cead0f28c7c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.106726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88883da-d9b8-4a60-95f4-cead0f28c7c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.108829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88883da-d9b8-4a60-95f4-cead0f28c7c8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.124182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64221eb-43e6-4fe6-9874-8cf7876ec069 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.129405 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78049f17-fb97-4821-9ae8-1801f4ac476a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.130158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78049f17-fb97-4821-9ae8-1801f4ac476a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.130377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78049f17-fb97-4821-9ae8-1801f4ac476a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.130712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78049f17-fb97-4821-9ae8-1801f4ac476a admin admin] PATH_INFO: `/v3/projects/097e46a08d19498aadf0ef0cd38c3cf7` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.132026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78049f17-fb97-4821-9ae8-1801f4ac476a admin admin] RBAC: Authorizing `identity:delete_project(project_id=097e46a08d19498aadf0ef0cd38c3cf7)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.134225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78049f17-fb97-4821-9ae8-1801f4ac476a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.145577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1139/2269] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:31:09 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:31:09.149747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88883da-d9b8-4a60-95f4-cead0f28c7c8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.149908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:09.194793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-78049f17-fb97-4821-9ae8-1801f4ac476a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '097e46a08d19498aadf0ef0cd38c3cf7', 'request_id': 'req-78049f17-fb97-4821-9ae8-1801f4ac476a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:09.199726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:09.204585 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-78049f17-fb97-4821-9ae8-1801f4ac476a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '097e46a08d19498aadf0ef0cd38c3cf7', 'request_id': 'req-78049f17-fb97-4821-9ae8-1801f4ac476a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:09.211948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-78049f17-fb97-4821-9ae8-1801f4ac476a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '097e46a08d19498aadf0ef0cd38c3cf7', 'request_id': 'req-78049f17-fb97-4821-9ae8-1801f4ac476a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:09.223006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88883da-d9b8-4a60-95f4-cead0f28c7c8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.225618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-78049f17-fb97-4821-9ae8-1801f4ac476a admin admin] The token cache is being invalidate because project 097e46a08d19498aadf0ef0cd38c3cf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:31:09.225908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-78049f17-fb97-4821-9ae8-1801f4ac476a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:09.226718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-78049f17-fb97-4821-9ae8-1801f4ac476a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:09.228465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1131/2270] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:31:09 2022] DELETE /identity/v3/projects/097e46a08d19498aadf0ef0cd38c3cf7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:31:09.233289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1917483-6a00-4983-a7c1-7c405c38fdd1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.233567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1917483-6a00-4983-a7c1-7c405c38fdd1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.233844 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1917483-6a00-4983-a7c1-7c405c38fdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.253950 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c1917483-6a00-4983-a7c1-7c405c38fdd1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:09.255260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c1917483-6a00-4983-a7c1-7c405c38fdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:09.304232 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020bdaa-ee64-4456-8e19-712c827fad78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.304851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020bdaa-ee64-4456-8e19-712c827fad78 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.305229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020bdaa-ee64-4456-8e19-712c827fad78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.305796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020bdaa-ee64-4456-8e19-712c827fad78 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.306450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020bdaa-ee64-4456-8e19-712c827fad78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.309088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020bdaa-ee64-4456-8e19-712c827fad78 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.334122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1140/2271] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:31:09 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:31:09.336265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1132/2272] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:31:09.338422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020bdaa-ee64-4456-8e19-712c827fad78 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.344355 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9020bdaa-ee64-4456-8e19-712c827fad78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:09.345517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe537b4-c886-4c40-bf46-952e3944e8b3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.345871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe537b4-c886-4c40-bf46-952e3944e8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.346216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe537b4-c886-4c40-bf46-952e3944e8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.357382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020bdaa-ee64-4456-8e19-712c827fad78 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.362191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acf5c3-3668-4e59-8576-df31abb6720e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.362801 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acf5c3-3668-4e59-8576-df31abb6720e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.363096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acf5c3-3668-4e59-8576-df31abb6720e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.363372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acf5c3-3668-4e59-8576-df31abb6720e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.364073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acf5c3-3668-4e59-8576-df31abb6720e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.365694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acf5c3-3668-4e59-8576-df31abb6720e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.367652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cbe537b4-c886-4c40-bf46-952e3944e8b3 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:09.369094 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cbe537b4-c886-4c40-bf46-952e3944e8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:09.378234 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1141/2273] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:31:09.380583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1133/2274] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:31:09.382165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acf5c3-3668-4e59-8576-df31abb6720e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.384701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe537b4-c886-4c40-bf46-952e3944e8b3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.393063 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acf5c3-3668-4e59-8576-df31abb6720e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.396976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe537b4-c886-4c40-bf46-952e3944e8b3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.398810 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7572b-78f3-4885-8d4d-e5e1b9522305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.399459 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7572b-78f3-4885-8d4d-e5e1b9522305 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.399740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7572b-78f3-4885-8d4d-e5e1b9522305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.400012 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7572b-78f3-4885-8d4d-e5e1b9522305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.400672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7572b-78f3-4885-8d4d-e5e1b9522305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.401678 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13bffb2-252d-43db-bbd0-4e40ed7f5897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.402288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13bffb2-252d-43db-bbd0-4e40ed7f5897 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.402414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7572b-78f3-4885-8d4d-e5e1b9522305 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.402558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13bffb2-252d-43db-bbd0-4e40ed7f5897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.402833 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13bffb2-252d-43db-bbd0-4e40ed7f5897 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.403485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13bffb2-252d-43db-bbd0-4e40ed7f5897 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:31:09.403811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13bffb2-252d-43db-bbd0-4e40ed7f5897 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.405403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13bffb2-252d-43db-bbd0-4e40ed7f5897 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.414174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1142/2275] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:31:09.417873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1134/2276] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:31:09.418438 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7572b-78f3-4885-8d4d-e5e1b9522305 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.421685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13bffb2-252d-43db-bbd0-4e40ed7f5897 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.428198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7572b-78f3-4885-8d4d-e5e1b9522305 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.431984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13bffb2-252d-43db-bbd0-4e40ed7f5897 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.433409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff70728-14cb-4c57-b1c8-16061729de31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.434087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff70728-14cb-4c57-b1c8-16061729de31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.434464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff70728-14cb-4c57-b1c8-16061729de31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.434738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff70728-14cb-4c57-b1c8-16061729de31 admin admin] PATH_INFO: `/v3/projects/b266bbdcc18a47d99b032f526ff467da/users/c08c63dbb1ae4da68235f78ae7db4905/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.440183 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095fcac-25a3-4d4a-9ade-83b892ea120b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.441582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095fcac-25a3-4d4a-9ade-83b892ea120b admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.442022 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095fcac-25a3-4d4a-9ade-83b892ea120b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.442360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095fcac-25a3-4d4a-9ade-83b892ea120b admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.443069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b095fcac-25a3-4d4a-9ade-83b892ea120b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.445108 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b095fcac-25a3-4d4a-9ade-83b892ea120b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.457177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff70728-14cb-4c57-b1c8-16061729de31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b266bbdcc18a47d99b032f526ff467da, user_id=c08c63dbb1ae4da68235f78ae7db4905, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.459930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff70728-14cb-4c57-b1c8-16061729de31 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.474601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1143/2277] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:31:09 2022] PUT /identity/v3/projects/b266bbdcc18a47d99b032f526ff467da/users/c08c63dbb1ae4da68235f78ae7db4905/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:09.478596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1135/2278] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:31:09 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:31:09.482039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff70728-14cb-4c57-b1c8-16061729de31 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.484668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d7e46-f5a1-42b0-a73c-cb262a518350 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.484979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d7e46-f5a1-42b0-a73c-cb262a518350 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.485343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d7e46-f5a1-42b0-a73c-cb262a518350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.513148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-793d7e46-f5a1-42b0-a73c-cb262a518350 None None] MFA Rules not processed for user `c08c63dbb1ae4da68235f78ae7db4905`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:09.514564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-793d7e46-f5a1-42b0-a73c-cb262a518350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:09.551249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff70728-14cb-4c57-b1c8-16061729de31 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.569161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:09.590466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1136/2279] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:31:09.600597 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:09.608073 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d7e46-f5a1-42b0-a73c-cb262a518350 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.609512 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-793d7e46-f5a1-42b0-a73c-cb262a518350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:09.616811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9027c-a5b6-4cad-a8bf-8e447243d605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.617461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9027c-a5b6-4cad-a8bf-8e447243d605 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.617737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9027c-a5b6-4cad-a8bf-8e447243d605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.618003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9027c-a5b6-4cad-a8bf-8e447243d605 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.618629 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde9027c-a5b6-4cad-a8bf-8e447243d605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.620641 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde9027c-a5b6-4cad-a8bf-8e447243d605 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.642634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1144/2280] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:31:09 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:31:09.646956 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9027c-a5b6-4cad-a8bf-8e447243d605 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.656941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9027c-a5b6-4cad-a8bf-8e447243d605 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.661792 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295cf748-95d1-4491-826a-f61f7319acfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.662403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cf748-95d1-4491-826a-f61f7319acfc admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.662680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cf748-95d1-4491-826a-f61f7319acfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.662953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cf748-95d1-4491-826a-f61f7319acfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.663606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295cf748-95d1-4491-826a-f61f7319acfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.665266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295cf748-95d1-4491-826a-f61f7319acfc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.675887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1145/2281] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:31:09.679954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295cf748-95d1-4491-826a-f61f7319acfc admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.681967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d7e46-f5a1-42b0-a73c-cb262a518350 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.686607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1fdf5-b74c-40f5-9016-ba600b945af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.687575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1fdf5-b74c-40f5-9016-ba600b945af1 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.687854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1fdf5-b74c-40f5-9016-ba600b945af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.688224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1fdf5-b74c-40f5-9016-ba600b945af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.692411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295cf748-95d1-4491-826a-f61f7319acfc admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.698264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1fdf5-b74c-40f5-9016-ba600b945af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.699883 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1fdf5-b74c-40f5-9016-ba600b945af1 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.700675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17aade5-2d6e-49f6-88a3-ddead635dbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.701327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17aade5-2d6e-49f6-88a3-ddead635dbef admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.701600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17aade5-2d6e-49f6-88a3-ddead635dbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.701868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17aade5-2d6e-49f6-88a3-ddead635dbef admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.702517 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17aade5-2d6e-49f6-88a3-ddead635dbef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.704101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17aade5-2d6e-49f6-88a3-ddead635dbef admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.709036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1137/2282] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:31:09.717167 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1146/2283] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:31:09.720722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1fdf5-b74c-40f5-9016-ba600b945af1 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.728632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17aade5-2d6e-49f6-88a3-ddead635dbef admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.731939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1fdf5-b74c-40f5-9016-ba600b945af1 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.736972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf49755b-3270-4b6c-8382-13956fa83a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.737674 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49755b-3270-4b6c-8382-13956fa83a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.738033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49755b-3270-4b6c-8382-13956fa83a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.738321 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49755b-3270-4b6c-8382-13956fa83a27 admin admin] PATH_INFO: `/v3/projects/6619450eef5a4a5b8d3b6a45fe412788/users/5817b0d9e6904be2b0ea482746d3d70f/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.738544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17aade5-2d6e-49f6-88a3-ddead635dbef admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.745648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00ebf4-d9ee-4194-889d-456197c3e6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:09.746489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00ebf4-d9ee-4194-889d-456197c3e6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.747139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00ebf4-d9ee-4194-889d-456197c3e6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.747139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00ebf4-d9ee-4194-889d-456197c3e6f5 admin admin] PATH_INFO: `/v3/users/c08c63dbb1ae4da68235f78ae7db4905` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.748184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00ebf4-d9ee-4194-889d-456197c3e6f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08c63dbb1ae4da68235f78ae7db4905)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.750134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00ebf4-d9ee-4194-889d-456197c3e6f5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.754112 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf49755b-3270-4b6c-8382-13956fa83a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6619450eef5a4a5b8d3b6a45fe412788, user_id=5817b0d9e6904be2b0ea482746d3d70f, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:09.756857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf49755b-3270-4b6c-8382-13956fa83a27 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:09.771153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1138/2284] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:31:09 2022] PUT /identity/v3/projects/6619450eef5a4a5b8d3b6a45fe412788/users/5817b0d9e6904be2b0ea482746d3d70f/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:09.782977 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41d745-12c7-4aa0-a95d-40c194ffd80b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.783657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41d745-12c7-4aa0-a95d-40c194ffd80b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.783930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41d745-12c7-4aa0-a95d-40c194ffd80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.805365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e00ebf4-d9ee-4194-889d-456197c3e6f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08c63dbb1ae4da68235f78ae7db4905'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:09.812577 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2f41d745-12c7-4aa0-a95d-40c194ffd80b None None] MFA Rules not processed for user `5817b0d9e6904be2b0ea482746d3d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:09.813342 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e00ebf4-d9ee-4194-889d-456197c3e6f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08c63dbb1ae4da68235f78ae7db4905'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:09.813926 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2f41d745-12c7-4aa0-a95d-40c194ffd80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:09.814081 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e00ebf4-d9ee-4194-889d-456197c3e6f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08c63dbb1ae4da68235f78ae7db4905'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:09.821881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e00ebf4-d9ee-4194-889d-456197c3e6f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08c63dbb1ae4da68235f78ae7db4905'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:09.833543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e00ebf4-d9ee-4194-889d-456197c3e6f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08c63dbb1ae4da68235f78ae7db4905'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:09.839649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1147/2285] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:31:09 2022] DELETE /identity/v3/users/c08c63dbb1ae4da68235f78ae7db4905 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:09.844982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458a266-479a-4fc7-82fb-9928b29bffa6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:09.845575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458a266-479a-4fc7-82fb-9928b29bffa6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:09.845848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458a266-479a-4fc7-82fb-9928b29bffa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:09.867149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7458a266-479a-4fc7-82fb-9928b29bffa6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:09.868382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7458a266-479a-4fc7-82fb-9928b29bffa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:09.894280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1139/2286] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:31:09.903074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41d745-12c7-4aa0-a95d-40c194ffd80b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.905027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2f41d745-12c7-4aa0-a95d-40c194ffd80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:09.932276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1148/2287] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:31:09.942525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458a266-479a-4fc7-82fb-9928b29bffa6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:09.945404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7458a266-479a-4fc7-82fb-9928b29bffa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:09.966684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:09.982106 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41d745-12c7-4aa0-a95d-40c194ffd80b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:09.991296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:10.017667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458a266-479a-4fc7-82fb-9928b29bffa6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:10.021466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb8ba1-2ffe-4232-a6f3-050cd0cdc5bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:10.022189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb8ba1-2ffe-4232-a6f3-050cd0cdc5bf service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:10.022977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb8ba1-2ffe-4232-a6f3-050cd0cdc5bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:10.022977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb8ba1-2ffe-4232-a6f3-050cd0cdc5bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:10.041888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb8ba1-2ffe-4232-a6f3-050cd0cdc5bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:10.043478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb8ba1-2ffe-4232-a6f3-050cd0cdc5bf service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:10.065453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c100e-03ab-4b51-bb07-89f3a57ef0bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:10.065453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c100e-03ab-4b51-bb07-89f3a57ef0bf service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:10.065453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c100e-03ab-4b51-bb07-89f3a57ef0bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:10.065453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c100e-03ab-4b51-bb07-89f3a57ef0bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:10.072071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1149/2288] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:10.137993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c100e-03ab-4b51-bb07-89f3a57ef0bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:10.139583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c100e-03ab-4b51-bb07-89f3a57ef0bf service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:10.151498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1140/2289] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:10.372824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb8ba1-2ffe-4232-a6f3-050cd0cdc5bf service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:10.374454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d7cb8ba1-2ffe-4232-a6f3-050cd0cdc5bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:10.388855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb8ba1-2ffe-4232-a6f3-050cd0cdc5bf service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:10.395359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37f194-c6ec-4bed-904a-af029b38f57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:10.396152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37f194-c6ec-4bed-904a-af029b38f57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:10.396920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37f194-c6ec-4bed-904a-af029b38f57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:10.397223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37f194-c6ec-4bed-904a-af029b38f57d admin admin] PATH_INFO: `/v3/projects/b266bbdcc18a47d99b032f526ff467da` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:10.398412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37f194-c6ec-4bed-904a-af029b38f57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b266bbdcc18a47d99b032f526ff467da)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:10.401386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37f194-c6ec-4bed-904a-af029b38f57d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:10.462817 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-db37f194-c6ec-4bed-904a-af029b38f57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b266bbdcc18a47d99b032f526ff467da', 'request_id': 'req-db37f194-c6ec-4bed-904a-af029b38f57d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:10.472550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-db37f194-c6ec-4bed-904a-af029b38f57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b266bbdcc18a47d99b032f526ff467da', 'request_id': 'req-db37f194-c6ec-4bed-904a-af029b38f57d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:10.481031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-db37f194-c6ec-4bed-904a-af029b38f57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b266bbdcc18a47d99b032f526ff467da', 'request_id': 'req-db37f194-c6ec-4bed-904a-af029b38f57d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:10.492930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-db37f194-c6ec-4bed-904a-af029b38f57d admin admin] The token cache is being invalidate because project b266bbdcc18a47d99b032f526ff467da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:31:10.493259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-db37f194-c6ec-4bed-904a-af029b38f57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:10.493916 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-db37f194-c6ec-4bed-904a-af029b38f57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:10.495446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1150/2290] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:31:10 2022] DELETE /identity/v3/projects/b266bbdcc18a47d99b032f526ff467da => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:10.500873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e71171-c67d-4737-91e7-8ac652b7ebbb None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:10.501323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e71171-c67d-4737-91e7-8ac652b7ebbb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:10.501499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e71171-c67d-4737-91e7-8ac652b7ebbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:10.522704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e1e71171-c67d-4737-91e7-8ac652b7ebbb None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:10.523936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e1e71171-c67d-4737-91e7-8ac652b7ebbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:10.584649 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1141/2291] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:10.592831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960325f4-7d94-4990-b67e-357a795b5d19 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:10.593174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960325f4-7d94-4990-b67e-357a795b5d19 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:10.593478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960325f4-7d94-4990-b67e-357a795b5d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:10.615585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-960325f4-7d94-4990-b67e-357a795b5d19 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:10.616814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-960325f4-7d94-4990-b67e-357a795b5d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:10.626645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1151/2292] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:10.630959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e71171-c67d-4737-91e7-8ac652b7ebbb None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:10.640872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e71171-c67d-4737-91e7-8ac652b7ebbb None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:10.646328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184cd9c-1b53-4344-b972-0780c633487d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:10.646920 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184cd9c-1b53-4344-b972-0780c633487d admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:10.647188 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184cd9c-1b53-4344-b972-0780c633487d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:10.647452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184cd9c-1b53-4344-b972-0780c633487d admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:10.648075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184cd9c-1b53-4344-b972-0780c633487d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:31:10.648388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184cd9c-1b53-4344-b972-0780c633487d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:10.650114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184cd9c-1b53-4344-b972-0780c633487d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:10.662092 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1142/2293] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:31:10.666652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960325f4-7d94-4990-b67e-357a795b5d19 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:10.676365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960325f4-7d94-4990-b67e-357a795b5d19 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:10.681665 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e7c6d-f587-419b-ae7f-02cca05b03cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:10.682248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e7c6d-f587-419b-ae7f-02cca05b03cd admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:10.682525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e7c6d-f587-419b-ae7f-02cca05b03cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:10.682909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e7c6d-f587-419b-ae7f-02cca05b03cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:10.683556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e7c6d-f587-419b-ae7f-02cca05b03cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:10.685671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e7c6d-f587-419b-ae7f-02cca05b03cd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:10.714251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1152/2294] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:31:10 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:31:10.718792 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184cd9c-1b53-4344-b972-0780c633487d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:10.729415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184cd9c-1b53-4344-b972-0780c633487d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:10.780113 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:10.788958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f015a-843e-45b3-9743-b0d5db8fb4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:10.789632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f015a-843e-45b3-9743-b0d5db8fb4df admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:10.789908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f015a-843e-45b3-9743-b0d5db8fb4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:10.790174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f015a-843e-45b3-9743-b0d5db8fb4df admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:10.790815 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f015a-843e-45b3-9743-b0d5db8fb4df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:10.792756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f015a-843e-45b3-9743-b0d5db8fb4df admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:10.816622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1143/2295] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:31:10 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:31:10.820992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e7c6d-f587-419b-ae7f-02cca05b03cd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:10.831694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e7c6d-f587-419b-ae7f-02cca05b03cd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:10.836555 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afe877-f1bc-4197-bb8d-60da49255fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:10.837183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afe877-f1bc-4197-bb8d-60da49255fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:10.837447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afe877-f1bc-4197-bb8d-60da49255fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:10.837724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afe877-f1bc-4197-bb8d-60da49255fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:10.838355 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afe877-f1bc-4197-bb8d-60da49255fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:10.839972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afe877-f1bc-4197-bb8d-60da49255fc7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:10.850460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1153/2296] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:31:10.854834 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f015a-843e-45b3-9743-b0d5db8fb4df admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:10.866764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f015a-843e-45b3-9743-b0d5db8fb4df admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:10.878040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc47dd4-386e-4d44-830d-bdaf2ac9fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:10.878863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc47dd4-386e-4d44-830d-bdaf2ac9fbdb admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:10.879154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc47dd4-386e-4d44-830d-bdaf2ac9fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:10.879543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc47dd4-386e-4d44-830d-bdaf2ac9fbdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:10.880211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc47dd4-386e-4d44-830d-bdaf2ac9fbdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:10.881962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc47dd4-386e-4d44-830d-bdaf2ac9fbdb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:10.896060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1144/2297] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:31:10.900345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afe877-f1bc-4197-bb8d-60da49255fc7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:10.910831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afe877-f1bc-4197-bb8d-60da49255fc7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:10.915948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e7ee6-95bc-42df-83b9-ed59f7a5a2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:10.916552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e7ee6-95bc-42df-83b9-ed59f7a5a2c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:10.916829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e7ee6-95bc-42df-83b9-ed59f7a5a2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:10.917696 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e7ee6-95bc-42df-83b9-ed59f7a5a2c7 admin admin] PATH_INFO: `/v3/projects/5f7466e0c0df480ba9fa6b7dcc8c300b/users/f563de69b21440378c51f04f74ee351b/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:10.933899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e7ee6-95bc-42df-83b9-ed59f7a5a2c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7466e0c0df480ba9fa6b7dcc8c300b, user_id=f563de69b21440378c51f04f74ee351b, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:10.936625 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e7ee6-95bc-42df-83b9-ed59f7a5a2c7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:10.952898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1154/2298] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:31:10 2022] PUT /identity/v3/projects/5f7466e0c0df480ba9fa6b7dcc8c300b/users/f563de69b21440378c51f04f74ee351b/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:10.965070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a419ce-0bf2-4a82-b142-089514d3eeb5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:10.965360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a419ce-0bf2-4a82-b142-089514d3eeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:10.965634 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a419ce-0bf2-4a82-b142-089514d3eeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:10.995605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d2a419ce-0bf2-4a82-b142-089514d3eeb5 None None] MFA Rules not processed for user `f563de69b21440378c51f04f74ee351b`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:10.996829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d2a419ce-0bf2-4a82-b142-089514d3eeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:11.064350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1145/2299] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:11.076520 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e7ee6-95bc-42df-83b9-ed59f7a5a2c7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:11.078591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-de1e7ee6-95bc-42df-83b9-ed59f7a5a2c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:11.132911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:11.149202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e7ee6-95bc-42df-83b9-ed59f7a5a2c7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:11.153536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ad064-866c-4026-8c55-ef78a4ec0483 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:11.154143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ad064-866c-4026-8c55-ef78a4ec0483 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:11.154417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ad064-866c-4026-8c55-ef78a4ec0483 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:11.154683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ad064-866c-4026-8c55-ef78a4ec0483 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:11.163409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ad064-866c-4026-8c55-ef78a4ec0483 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:11.164841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ad064-866c-4026-8c55-ef78a4ec0483 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:11.177233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1155/2300] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:11.196652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a419ce-0bf2-4a82-b142-089514d3eeb5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:11.275594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a419ce-0bf2-4a82-b142-089514d3eeb5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:11.280985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:11.281700 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:11.281978 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:11.282243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] PATH_INFO: `/v3/users/f563de69b21440378c51f04f74ee351b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:11.283406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=f563de69b21440378c51f04f74ee351b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:11.285333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:11.336648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f563de69b21440378c51f04f74ee351b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:11.345180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f563de69b21440378c51f04f74ee351b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:11.345698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f563de69b21440378c51f04f74ee351b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:11.355257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f563de69b21440378c51f04f74ee351b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:11.357299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:11.371696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f563de69b21440378c51f04f74ee351b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:11.377961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1146/2301] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:31:11 2022] DELETE /identity/v3/users/f563de69b21440378c51f04f74ee351b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:11.382731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916bb8b-3cdc-4287-85ac-ff176167fffb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:11.383018 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916bb8b-3cdc-4287-85ac-ff176167fffb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:11.383298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916bb8b-3cdc-4287-85ac-ff176167fffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:11.410324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1916bb8b-3cdc-4287-85ac-ff176167fffb None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:11.411717 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1916bb8b-3cdc-4287-85ac-ff176167fffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:11.477937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1156/2302] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:31:11.488973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:11.490497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:11.570085 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a180a-f88b-41ca-9a41-49e7bea20cae admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:11.574070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f527f5-6dfb-46c1-89b0-6fbe89a5fb1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:11.574722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f527f5-6dfb-46c1-89b0-6fbe89a5fb1d service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:11.576793 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f527f5-6dfb-46c1-89b0-6fbe89a5fb1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:11.577163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f527f5-6dfb-46c1-89b0-6fbe89a5fb1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:11.591035 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f527f5-6dfb-46c1-89b0-6fbe89a5fb1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:11.592529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f527f5-6dfb-46c1-89b0-6fbe89a5fb1d service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:11.603744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1147/2303] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:11.880498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916bb8b-3cdc-4287-85ac-ff176167fffb None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:11.881975 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1916bb8b-3cdc-4287-85ac-ff176167fffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:11.894113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916bb8b-3cdc-4287-85ac-ff176167fffb None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:11.899906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:11.900677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:11.900951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:11.901291 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d admin admin] PATH_INFO: `/v3/projects/5f7466e0c0df480ba9fa6b7dcc8c300b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:11.902467 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f7466e0c0df480ba9fa6b7dcc8c300b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:11.904601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:11.956003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f7466e0c0df480ba9fa6b7dcc8c300b', 'request_id': 'req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:11.964365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f7466e0c0df480ba9fa6b7dcc8c300b', 'request_id': 'req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:11.970678 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f7466e0c0df480ba9fa6b7dcc8c300b', 'request_id': 'req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:11.982548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d admin admin] The token cache is being invalidate because project 5f7466e0c0df480ba9fa6b7dcc8c300b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:31:11.982832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:11.983456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6c7c95fa-09b8-4c81-8b9c-6e72a40d6e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:11.984983 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1157/2304] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:31:11 2022] DELETE /identity/v3/projects/5f7466e0c0df480ba9fa6b7dcc8c300b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:11.990272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f97d9-5f00-4427-a47b-f1ba41a88219 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:11.990767 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f97d9-5f00-4427-a47b-f1ba41a88219 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:11.990862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f97d9-5f00-4427-a47b-f1ba41a88219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:12.012145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a72f97d9-5f00-4427-a47b-f1ba41a88219 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:12.013332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a72f97d9-5f00-4427-a47b-f1ba41a88219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:12.077599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1148/2305] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:12.085694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331468b-79a1-4186-8481-1230c8410768 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:12.086003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331468b-79a1-4186-8481-1230c8410768 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:12.086276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331468b-79a1-4186-8481-1230c8410768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:12.108682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0331468b-79a1-4186-8481-1230c8410768 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:12.110034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0331468b-79a1-4186-8481-1230c8410768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:12.119901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1158/2306] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:12.124231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f97d9-5f00-4427-a47b-f1ba41a88219 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:12.134493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f97d9-5f00-4427-a47b-f1ba41a88219 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:12.139487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d39e77-3ea8-4cf4-97dd-58d902bb8b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:12.140103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d39e77-3ea8-4cf4-97dd-58d902bb8b91 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:12.140372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d39e77-3ea8-4cf4-97dd-58d902bb8b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:12.140647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d39e77-3ea8-4cf4-97dd-58d902bb8b91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:12.141320 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d39e77-3ea8-4cf4-97dd-58d902bb8b91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:31:12.141621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d39e77-3ea8-4cf4-97dd-58d902bb8b91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:12.143213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d39e77-3ea8-4cf4-97dd-58d902bb8b91 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:12.155072 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1149/2307] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:31:12.159529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331468b-79a1-4186-8481-1230c8410768 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:12.169149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331468b-79a1-4186-8481-1230c8410768 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:12.174190 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e0ebd-82e0-42fc-a073-298fa31e8a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:12.174796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0ebd-82e0-42fc-a073-298fa31e8a12 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:12.175070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0ebd-82e0-42fc-a073-298fa31e8a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:12.175338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0ebd-82e0-42fc-a073-298fa31e8a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:12.175965 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e0ebd-82e0-42fc-a073-298fa31e8a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:12.177882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e0ebd-82e0-42fc-a073-298fa31e8a12 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:12.210694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1159/2308] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:31:12 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:31:12.215193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d39e77-3ea8-4cf4-97dd-58d902bb8b91 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:12.226817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d39e77-3ea8-4cf4-97dd-58d902bb8b91 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:12.295459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852c7f7-311e-445e-a9aa-4541e7a45f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:12.296153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852c7f7-311e-445e-a9aa-4541e7a45f4d admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:12.296494 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852c7f7-311e-445e-a9aa-4541e7a45f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:12.296854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852c7f7-311e-445e-a9aa-4541e7a45f4d admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:12.297549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852c7f7-311e-445e-a9aa-4541e7a45f4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:12.299538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852c7f7-311e-445e-a9aa-4541e7a45f4d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:12.335827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1150/2309] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:31:12 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:31:12.337408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e0ebd-82e0-42fc-a073-298fa31e8a12 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:12.350165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e0ebd-82e0-42fc-a073-298fa31e8a12 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:12.356280 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d7d40-b057-4ab0-88e4-e6382d30bf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:12.356996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d7d40-b057-4ab0-88e4-e6382d30bf8f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:12.357374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d7d40-b057-4ab0-88e4-e6382d30bf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:12.357710 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d7d40-b057-4ab0-88e4-e6382d30bf8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:12.358457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d7d40-b057-4ab0-88e4-e6382d30bf8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:12.360262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d7d40-b057-4ab0-88e4-e6382d30bf8f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:12.373156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1160/2310] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:31:12.375947 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852c7f7-311e-445e-a9aa-4541e7a45f4d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:12.381531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:12.389026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852c7f7-311e-445e-a9aa-4541e7a45f4d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:12.395582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7167af-140b-41ef-8ee6-4760acf24845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:12.396156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7167af-140b-41ef-8ee6-4760acf24845 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:12.396887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7167af-140b-41ef-8ee6-4760acf24845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:12.397207 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7167af-140b-41ef-8ee6-4760acf24845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:12.397910 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7167af-140b-41ef-8ee6-4760acf24845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:12.400012 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7167af-140b-41ef-8ee6-4760acf24845 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:12.414942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1151/2311] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:31:12.419753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d7d40-b057-4ab0-88e4-e6382d30bf8f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:12.434318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d7d40-b057-4ab0-88e4-e6382d30bf8f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:12.441228 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c59ba7-3cf5-49bd-b7c6-abe9f48fb0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:12.442104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c59ba7-3cf5-49bd-b7c6-abe9f48fb0ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:12.443464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c59ba7-3cf5-49bd-b7c6-abe9f48fb0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:12.443464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c59ba7-3cf5-49bd-b7c6-abe9f48fb0ce admin admin] PATH_INFO: `/v3/projects/7b231192d1d3410fb98e63662857895a/users/d70c324ad3124ba79fded83fe5d79627/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:12.459867 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c59ba7-3cf5-49bd-b7c6-abe9f48fb0ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b231192d1d3410fb98e63662857895a, user_id=d70c324ad3124ba79fded83fe5d79627, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:12.479889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c59ba7-3cf5-49bd-b7c6-abe9f48fb0ce admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:12.485671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1161/2312] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:31:12 2022] PUT /identity/v3/projects/7b231192d1d3410fb98e63662857895a/users/d70c324ad3124ba79fded83fe5d79627/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:31:12.496845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a55d04-6ef4-4727-837f-cbf8c9d83ae1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:12.497158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a55d04-6ef4-4727-837f-cbf8c9d83ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:12.497435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a55d04-6ef4-4727-837f-cbf8c9d83ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:12.530128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a5a55d04-6ef4-4727-837f-cbf8c9d83ae1 None None] MFA Rules not processed for user `d70c324ad3124ba79fded83fe5d79627`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:12.531553 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a5a55d04-6ef4-4727-837f-cbf8c9d83ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:12.608159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1152/2313] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:31:12.622592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c59ba7-3cf5-49bd-b7c6-abe9f48fb0ce admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:12.624614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e3c59ba7-3cf5-49bd-b7c6-abe9f48fb0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:12.632959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:12.640668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e0399-c986-4bed-a13f-4fc38fb2bddb None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:12.641041 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e0399-c986-4bed-a13f-4fc38fb2bddb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:12.641420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e0399-c986-4bed-a13f-4fc38fb2bddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:12.668692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-3a7e0399-c986-4bed-a13f-4fc38fb2bddb None None] MFA Rules not processed for user `5817b0d9e6904be2b0ea482746d3d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:12.670050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3a7e0399-c986-4bed-a13f-4fc38fb2bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:12.706629 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c59ba7-3cf5-49bd-b7c6-abe9f48fb0ce admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:12.711751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f192b-7aec-4c12-aafe-75ee12a32473 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:12.712539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f192b-7aec-4c12-aafe-75ee12a32473 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:12.712963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f192b-7aec-4c12-aafe-75ee12a32473 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:12.713488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f192b-7aec-4c12-aafe-75ee12a32473 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:12.723678 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f192b-7aec-4c12-aafe-75ee12a32473 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:12.727398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f192b-7aec-4c12-aafe-75ee12a32473 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:12.739845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1153/2314] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:12.742613 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1162/2315] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:12.746242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e0399-c986-4bed-a13f-4fc38fb2bddb None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:12.748051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3a7e0399-c986-4bed-a13f-4fc38fb2bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:12.815834 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e0399-c986-4bed-a13f-4fc38fb2bddb None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:12.818080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98fda0-8782-459f-b1d3-fabee63d2337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:12.818751 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98fda0-8782-459f-b1d3-fabee63d2337 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:12.819028 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98fda0-8782-459f-b1d3-fabee63d2337 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:12.819395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98fda0-8782-459f-b1d3-fabee63d2337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:12.821573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-df98fda0-8782-459f-b1d3-fabee63d2337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:12.871561 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:12.899114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98fda0-8782-459f-b1d3-fabee63d2337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:12.900549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98fda0-8782-459f-b1d3-fabee63d2337 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:12.912110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1154/2316] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:31:22.736019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=24 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:22.906758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=11 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:46.023511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f192b-7aec-4c12-aafe-75ee12a32473 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:46.025015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5f2f192b-7aec-4c12-aafe-75ee12a32473 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:46.096743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f192b-7aec-4c12-aafe-75ee12a32473 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:46.102159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295108b-5b60-4c3b-b623-2c0980f1e3c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:46.103808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295108b-5b60-4c3b-b623-2c0980f1e3c7 service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:46.104085 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295108b-5b60-4c3b-b623-2c0980f1e3c7 service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:46.104354 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295108b-5b60-4c3b-b623-2c0980f1e3c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:46.106149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1295108b-5b60-4c3b-b623-2c0980f1e3c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:46.118081 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295108b-5b60-4c3b-b623-2c0980f1e3c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:46.119692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295108b-5b60-4c3b-b623-2c0980f1e3c7 service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:46.131319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1163/2317] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:47.479403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98fda0-8782-459f-b1d3-fabee63d2337 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:47.557236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98fda0-8782-459f-b1d3-fabee63d2337 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:47.561983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:47.562633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:47.562911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:47.563182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] PATH_INFO: `/v3/users/d70c324ad3124ba79fded83fe5d79627` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:47.564448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d70c324ad3124ba79fded83fe5d79627)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:47.566910 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:47.630132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd70c324ad3124ba79fded83fe5d79627'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:47.639390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd70c324ad3124ba79fded83fe5d79627'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:47.640155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd70c324ad3124ba79fded83fe5d79627'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:47.648828 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd70c324ad3124ba79fded83fe5d79627'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:47.662995 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd70c324ad3124ba79fded83fe5d79627'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:47.670364 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1155/2318] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:31:47 2022] DELETE /identity/v3/users/d70c324ad3124ba79fded83fe5d79627 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:47.674524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b4c2c-41f3-4f2c-a552-bd694d81c67c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:47.675013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b4c2c-41f3-4f2c-a552-bd694d81c67c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:47.675561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b4c2c-41f3-4f2c-a552-bd694d81c67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:47.694840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9e5b4c2c-41f3-4f2c-a552-bd694d81c67c None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:47.696053 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9e5b4c2c-41f3-4f2c-a552-bd694d81c67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:47.762144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1164/2319] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:31:47.772170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:47.773640 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:47.830591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3754f-95de-476a-847a-8c8c64c71813 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:47.834883 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fbcdf-6709-49a5-a146-71ebe8fe4337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:47.835494 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fbcdf-6709-49a5-a146-71ebe8fe4337 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:47.835765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fbcdf-6709-49a5-a146-71ebe8fe4337 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:47.836032 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fbcdf-6709-49a5-a146-71ebe8fe4337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:47.846059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2fbcdf-6709-49a5-a146-71ebe8fe4337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:47.847479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2fbcdf-6709-49a5-a146-71ebe8fe4337 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:47.857933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1156/2320] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:48.175786 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b4c2c-41f3-4f2c-a552-bd694d81c67c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:48.177578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9e5b4c2c-41f3-4f2c-a552-bd694d81c67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:48.190123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b4c2c-41f3-4f2c-a552-bd694d81c67c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:48.195818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6253135-1261-4d42-bd39-fe8a0c3436b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:48.196458 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6253135-1261-4d42-bd39-fe8a0c3436b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:48.196737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6253135-1261-4d42-bd39-fe8a0c3436b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:48.197006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6253135-1261-4d42-bd39-fe8a0c3436b9 admin admin] PATH_INFO: `/v3/projects/7b231192d1d3410fb98e63662857895a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:48.198283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6253135-1261-4d42-bd39-fe8a0c3436b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b231192d1d3410fb98e63662857895a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:48.200276 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6253135-1261-4d42-bd39-fe8a0c3436b9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:48.249976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e6253135-1261-4d42-bd39-fe8a0c3436b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b231192d1d3410fb98e63662857895a', 'request_id': 'req-e6253135-1261-4d42-bd39-fe8a0c3436b9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:48.258622 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e6253135-1261-4d42-bd39-fe8a0c3436b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b231192d1d3410fb98e63662857895a', 'request_id': 'req-e6253135-1261-4d42-bd39-fe8a0c3436b9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:48.267835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e6253135-1261-4d42-bd39-fe8a0c3436b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b231192d1d3410fb98e63662857895a', 'request_id': 'req-e6253135-1261-4d42-bd39-fe8a0c3436b9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:48.280340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e6253135-1261-4d42-bd39-fe8a0c3436b9 admin admin] The token cache is being invalidate because project 7b231192d1d3410fb98e63662857895a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:31:48.280625 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e6253135-1261-4d42-bd39-fe8a0c3436b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:48.281422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e6253135-1261-4d42-bd39-fe8a0c3436b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:48.291171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1165/2321] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:31:48 2022] DELETE /identity/v3/projects/7b231192d1d3410fb98e63662857895a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:52.643736 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdcdb9-4d3b-4a0a-acb2-505046fdf398 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:52.644262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdcdb9-4d3b-4a0a-acb2-505046fdf398 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:52.644262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdcdb9-4d3b-4a0a-acb2-505046fdf398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:52.664386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-eacdcdb9-4d3b-4a0a-acb2-505046fdf398 None None] MFA Rules not processed for user `ff241cbd3f7c47328ac5ef2ed4c66e18`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:52.665679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-eacdcdb9-4d3b-4a0a-acb2-505046fdf398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:52.683743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:52.725109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1157/2322] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:56.252861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3821c9-99f8-467e-ac8e-d0170b3122e9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:56.253469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3821c9-99f8-467e-ac8e-d0170b3122e9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:56.253469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3821c9-99f8-467e-ac8e-d0170b3122e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:56.275474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9e3821c9-99f8-467e-ac8e-d0170b3122e9 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:56.277633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9e3821c9-99f8-467e-ac8e-d0170b3122e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:56.289765 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:56.339394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1166/2323] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:31:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:56.385943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdcdb9-4d3b-4a0a-acb2-505046fdf398 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:56.387434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-eacdcdb9-4d3b-4a0a-acb2-505046fdf398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:56.459874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdcdb9-4d3b-4a0a-acb2-505046fdf398 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:56.464873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7051f1-aa2b-430b-8ba2-70bf3cdc23b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:56.465514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7051f1-aa2b-430b-8ba2-70bf3cdc23b9 service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:56.465790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7051f1-aa2b-430b-8ba2-70bf3cdc23b9 service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:56.466058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7051f1-aa2b-430b-8ba2-70bf3cdc23b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:56.474571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7051f1-aa2b-430b-8ba2-70bf3cdc23b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:56.476175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7051f1-aa2b-430b-8ba2-70bf3cdc23b9 service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:56.484453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1158/2324] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:31:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:31:57.120471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d8e6f-f9ca-4814-b4d0-23a4804d3d16 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:57.120972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d8e6f-f9ca-4814-b4d0-23a4804d3d16 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:57.121475 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d8e6f-f9ca-4814-b4d0-23a4804d3d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:57.158654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4c3d8e6f-f9ca-4814-b4d0-23a4804d3d16 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:57.160377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4c3d8e6f-f9ca-4814-b4d0-23a4804d3d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:57.169567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1167/2325] 10.222.0.60 () {58 vars in 967 bytes} [Sat Sep 17 04:31:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:57.187221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7051f1-aa2b-430b-8ba2-70bf3cdc23b9 service swift] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:57.199926 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7051f1-aa2b-430b-8ba2-70bf3cdc23b9 service swift] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:57.210586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c3f61-90f6-4913-b910-146dc97edac1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:57.211314 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c3f61-90f6-4913-b910-146dc97edac1 service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:57.211695 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c3f61-90f6-4913-b910-146dc97edac1 service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:57.212069 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c3f61-90f6-4913-b910-146dc97edac1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:57.221337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c3f61-90f6-4913-b910-146dc97edac1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:57.223728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c3f61-90f6-4913-b910-146dc97edac1 service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:57.233134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1159/2326] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:31:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:31:57.497975 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dcd0c-0279-47c8-ae13-7e7c923b4223 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:57.498467 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dcd0c-0279-47c8-ae13-7e7c923b4223 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:57.498467 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dcd0c-0279-47c8-ae13-7e7c923b4223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:57.518999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e85dcd0c-0279-47c8-ae13-7e7c923b4223 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:57.520126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e85dcd0c-0279-47c8-ae13-7e7c923b4223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:57.527683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1168/2327] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:31:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:57.539849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c3f61-90f6-4913-b910-146dc97edac1 service swift] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:57.550575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c3f61-90f6-4913-b910-146dc97edac1 service swift] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:57.556482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53541a6-aaa9-4013-89e6-49ffcc548021 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:57.557107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53541a6-aaa9-4013-89e6-49ffcc548021 service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:57.557391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53541a6-aaa9-4013-89e6-49ffcc548021 service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:57.557659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53541a6-aaa9-4013-89e6-49ffcc548021 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:57.565979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53541a6-aaa9-4013-89e6-49ffcc548021 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:57.567551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53541a6-aaa9-4013-89e6-49ffcc548021 service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:57.575692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1160/2328] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:31:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:31:57.905358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8a3a0-ae72-4a99-a369-43da8b668f86 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:57.905637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8a3a0-ae72-4a99-a369-43da8b668f86 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:57.905972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8a3a0-ae72-4a99-a369-43da8b668f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:57.925899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0dc8a3a0-ae72-4a99-a369-43da8b668f86 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:57.927088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0dc8a3a0-ae72-4a99-a369-43da8b668f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:57.934868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1169/2329] 10.222.0.60 () {58 vars in 967 bytes} [Sat Sep 17 04:31:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:57.947517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53541a6-aaa9-4013-89e6-49ffcc548021 service swift] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:57.958368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53541a6-aaa9-4013-89e6-49ffcc548021 service swift] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:57.964744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd6a90-ec4b-4a6e-8083-a4e2040373a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:57.965370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd6a90-ec4b-4a6e-8083-a4e2040373a6 service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:57.965647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd6a90-ec4b-4a6e-8083-a4e2040373a6 service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:57.965914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd6a90-ec4b-4a6e-8083-a4e2040373a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:57.974884 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd6a90-ec4b-4a6e-8083-a4e2040373a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:57.976509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd6a90-ec4b-4a6e-8083-a4e2040373a6 service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:57.984711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1161/2330] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:31:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:31:58.086572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8a3a0-ae72-4a99-a369-43da8b668f86 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:58.091713 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0dc8a3a0-ae72-4a99-a369-43da8b668f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:58.210253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8a3a0-ae72-4a99-a369-43da8b668f86 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:58.225612 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:58.227626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:58.229430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:58.229951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] PATH_INFO: `/v3/users/853d9a4f9d28459a82cc79186e233796` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:58.232379 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=853d9a4f9d28459a82cc79186e233796)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:58.235228 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:58.318998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '853d9a4f9d28459a82cc79186e233796'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:58.330399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '853d9a4f9d28459a82cc79186e233796'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:58.331541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '853d9a4f9d28459a82cc79186e233796'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:58.342734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '853d9a4f9d28459a82cc79186e233796'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:58.363437 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '853d9a4f9d28459a82cc79186e233796'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:58.373243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1170/2331] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:31:58 2022] DELETE /identity/v3/users/853d9a4f9d28459a82cc79186e233796 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:31:58.377890 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929c1f8-5efd-4ba7-a2ef-031ae54afb17 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:58.378169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929c1f8-5efd-4ba7-a2ef-031ae54afb17 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:58.378437 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929c1f8-5efd-4ba7-a2ef-031ae54afb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:58.409859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a929c1f8-5efd-4ba7-a2ef-031ae54afb17 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:58.412263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a929c1f8-5efd-4ba7-a2ef-031ae54afb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:58.493969 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1162/2332] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:58.504537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:58.506142 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:58.529037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:58.580207 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc4a77-c421-4a06-8e41-2dcbb5b05331 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:58.584568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3a42a-6342-4da1-82cc-769ddb1d5120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:58.584870 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3a42a-6342-4da1-82cc-769ddb1d5120 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:58.585206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3a42a-6342-4da1-82cc-769ddb1d5120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:58.585424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3a42a-6342-4da1-82cc-769ddb1d5120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:58.594858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3a42a-6342-4da1-82cc-769ddb1d5120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:58.596294 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3a42a-6342-4da1-82cc-769ddb1d5120 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:58.606014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1171/2333] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:31:58.906039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929c1f8-5efd-4ba7-a2ef-031ae54afb17 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:58.907710 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a929c1f8-5efd-4ba7-a2ef-031ae54afb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:58.922663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929c1f8-5efd-4ba7-a2ef-031ae54afb17 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:58.936549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a733d-c86a-4159-837e-bad8f20ed11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:58.937374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a733d-c86a-4159-837e-bad8f20ed11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:58.937647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a733d-c86a-4159-837e-bad8f20ed11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:58.937926 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a733d-c86a-4159-837e-bad8f20ed11c admin admin] PATH_INFO: `/v3/projects/7be428966f9641d794c310078b177dc5` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:58.939078 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a733d-c86a-4159-837e-bad8f20ed11c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be428966f9641d794c310078b177dc5)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:58.941052 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a733d-c86a-4159-837e-bad8f20ed11c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:58.956614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:59.004516 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a68a733d-c86a-4159-837e-bad8f20ed11c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be428966f9641d794c310078b177dc5', 'request_id': 'req-a68a733d-c86a-4159-837e-bad8f20ed11c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:59.014852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a68a733d-c86a-4159-837e-bad8f20ed11c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be428966f9641d794c310078b177dc5', 'request_id': 'req-a68a733d-c86a-4159-837e-bad8f20ed11c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:59.022087 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a68a733d-c86a-4159-837e-bad8f20ed11c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be428966f9641d794c310078b177dc5', 'request_id': 'req-a68a733d-c86a-4159-837e-bad8f20ed11c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:59.033805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a68a733d-c86a-4159-837e-bad8f20ed11c admin admin] The token cache is being invalidate because project 7be428966f9641d794c310078b177dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:31:59.034094 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a68a733d-c86a-4159-837e-bad8f20ed11c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:59.034866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a68a733d-c86a-4159-837e-bad8f20ed11c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:31:59.037293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1163/2334] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:31:58 2022] DELETE /identity/v3/projects/7be428966f9641d794c310078b177dc5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:59.042339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e5dc5-a84c-43e0-9ebd-7fa396c15d75 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:59.042689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e5dc5-a84c-43e0-9ebd-7fa396c15d75 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:59.043028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e5dc5-a84c-43e0-9ebd-7fa396c15d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:59.063514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-565e5dc5-a84c-43e0-9ebd-7fa396c15d75 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:59.064701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-565e5dc5-a84c-43e0-9ebd-7fa396c15d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:59.124967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1172/2335] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:31:59.132429 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76859a1b-9ce6-487e-8499-58a20a9392e3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:59.132790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76859a1b-9ce6-487e-8499-58a20a9392e3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:59.133155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76859a1b-9ce6-487e-8499-58a20a9392e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:59.154136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-76859a1b-9ce6-487e-8499-58a20a9392e3 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:59.155652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-76859a1b-9ce6-487e-8499-58a20a9392e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:59.165275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1164/2336] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:31:59.169773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e5dc5-a84c-43e0-9ebd-7fa396c15d75 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:59.179218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e5dc5-a84c-43e0-9ebd-7fa396c15d75 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:59.183867 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fb31c-f01e-40a7-bf5d-93b4a8477899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:59.184474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fb31c-f01e-40a7-bf5d-93b4a8477899 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:59.184751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fb31c-f01e-40a7-bf5d-93b4a8477899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:59.185021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fb31c-f01e-40a7-bf5d-93b4a8477899 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:59.185711 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20fb31c-f01e-40a7-bf5d-93b4a8477899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:31:59.186011 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20fb31c-f01e-40a7-bf5d-93b4a8477899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:59.187663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20fb31c-f01e-40a7-bf5d-93b4a8477899 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:59.200884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1173/2337] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:31:59.210630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76859a1b-9ce6-487e-8499-58a20a9392e3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:59.219870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76859a1b-9ce6-487e-8499-58a20a9392e3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:59.225394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7f3d5-ff07-494f-8e5c-14cece859e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:59.225989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7f3d5-ff07-494f-8e5c-14cece859e55 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:59.226264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7f3d5-ff07-494f-8e5c-14cece859e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:59.226527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7f3d5-ff07-494f-8e5c-14cece859e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:59.227135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7f3d5-ff07-494f-8e5c-14cece859e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:59.229024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7f3d5-ff07-494f-8e5c-14cece859e55 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:59.257801 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1165/2338] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:31:59 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:31:59.262170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fb31c-f01e-40a7-bf5d-93b4a8477899 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:59.272551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fb31c-f01e-40a7-bf5d-93b4a8477899 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:59.331504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f3947-68ce-4306-9336-67329b289581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:59.332075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f3947-68ce-4306-9336-67329b289581 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:59.332358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f3947-68ce-4306-9336-67329b289581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:59.332625 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f3947-68ce-4306-9336-67329b289581 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:59.333291 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f3947-68ce-4306-9336-67329b289581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:59.335219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f3947-68ce-4306-9336-67329b289581 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:59.366071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1174/2339] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:31:59 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:31:59.367120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7f3d5-ff07-494f-8e5c-14cece859e55 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:59.377730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7f3d5-ff07-494f-8e5c-14cece859e55 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:59.382635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c861755-82ec-4ea5-b08f-c58a881ca34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:59.383233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861755-82ec-4ea5-b08f-c58a881ca34d admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:59.383509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861755-82ec-4ea5-b08f-c58a881ca34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:59.383779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861755-82ec-4ea5-b08f-c58a881ca34d admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:59.384418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c861755-82ec-4ea5-b08f-c58a881ca34d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:59.386122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c861755-82ec-4ea5-b08f-c58a881ca34d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:59.397333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1166/2340] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:31:59.400903 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f3947-68ce-4306-9336-67329b289581 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:59.411646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f3947-68ce-4306-9336-67329b289581 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:59.416336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819b88c-a788-4a9c-85de-251aba2eb45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:59.416949 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819b88c-a788-4a9c-85de-251aba2eb45b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:59.417265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819b88c-a788-4a9c-85de-251aba2eb45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:59.417538 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819b88c-a788-4a9c-85de-251aba2eb45b admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:59.418261 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819b88c-a788-4a9c-85de-251aba2eb45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:59.419871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819b88c-a788-4a9c-85de-251aba2eb45b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:59.431767 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1175/2341] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:31:59.436126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c861755-82ec-4ea5-b08f-c58a881ca34d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:59.447267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c861755-82ec-4ea5-b08f-c58a881ca34d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:59.452357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31556261-4538-4a55-9da9-6652386a7e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:59.453039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31556261-4538-4a55-9da9-6652386a7e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:59.453332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31556261-4538-4a55-9da9-6652386a7e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:59.453665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31556261-4538-4a55-9da9-6652386a7e10 admin admin] PATH_INFO: `/v3/projects/35e627b87be74f679c84863eb37f9a6f/users/1195b16983ca4363abc1cd2829ecdee8/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:59.468719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31556261-4538-4a55-9da9-6652386a7e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e627b87be74f679c84863eb37f9a6f, user_id=1195b16983ca4363abc1cd2829ecdee8, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:59.471382 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31556261-4538-4a55-9da9-6652386a7e10 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:59.485667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1167/2342] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:31:59 2022] PUT /identity/v3/projects/35e627b87be74f679c84863eb37f9a6f/users/1195b16983ca4363abc1cd2829ecdee8/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:31:59.497103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd156fe-c371-4460-a003-c6fc1953f6a3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:59.497447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd156fe-c371-4460-a003-c6fc1953f6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:59.497703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd156fe-c371-4460-a003-c6fc1953f6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:59.526672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ecd156fe-c371-4460-a003-c6fc1953f6a3 None None] MFA Rules not processed for user `1195b16983ca4363abc1cd2829ecdee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:31:59.528044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ecd156fe-c371-4460-a003-c6fc1953f6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:59.549178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:59.630121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1176/2343] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:31:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:31:59.642861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31556261-4538-4a55-9da9-6652386a7e10 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:31:59.647463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-31556261-4538-4a55-9da9-6652386a7e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:31:59.719154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31556261-4538-4a55-9da9-6652386a7e10 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:31:59.722840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1c94a-6fb2-456a-b369-75fab69d722f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:31:59.723814 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1c94a-6fb2-456a-b369-75fab69d722f service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:31:59.724095 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1c94a-6fb2-456a-b369-75fab69d722f service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:31:59.724366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1c94a-6fb2-456a-b369-75fab69d722f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:31:59.733186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1c94a-6fb2-456a-b369-75fab69d722f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:31:59.734654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1c94a-6fb2-456a-b369-75fab69d722f service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:31:59.743761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1168/2344] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:31:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:32:00.247321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd156fe-c371-4460-a003-c6fc1953f6a3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:00.248907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ecd156fe-c371-4460-a003-c6fc1953f6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:00.314578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd156fe-c371-4460-a003-c6fc1953f6a3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:00.320676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:00.321356 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:00.321633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:00.321901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] PATH_INFO: `/v3/users/5817b0d9e6904be2b0ea482746d3d70f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:00.323210 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5817b0d9e6904be2b0ea482746d3d70f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:00.325196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:00.378533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5817b0d9e6904be2b0ea482746d3d70f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:00.386735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5817b0d9e6904be2b0ea482746d3d70f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:00.387352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5817b0d9e6904be2b0ea482746d3d70f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:00.394268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5817b0d9e6904be2b0ea482746d3d70f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:00.405803 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5817b0d9e6904be2b0ea482746d3d70f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:00.411883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1177/2345] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:32:00 2022] DELETE /identity/v3/users/5817b0d9e6904be2b0ea482746d3d70f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:32:00.417307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eaae26-2785-4702-8a02-58dbe21a1842 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:00.417810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eaae26-2785-4702-8a02-58dbe21a1842 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:00.417928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eaae26-2785-4702-8a02-58dbe21a1842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:00.439089 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a7eaae26-2785-4702-8a02-58dbe21a1842 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:32:00.440304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a7eaae26-2785-4702-8a02-58dbe21a1842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:00.474397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:00.502386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1169/2346] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:32:00.512386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:00.513904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:00.554816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:00.575156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213d225-2799-4563-b84c-df62ef97505e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:00.579117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06398f2-4fe1-4f76-b0cc-267a95e77f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:00.579754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06398f2-4fe1-4f76-b0cc-267a95e77f49 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:00.580034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06398f2-4fe1-4f76-b0cc-267a95e77f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:00.580299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06398f2-4fe1-4f76-b0cc-267a95e77f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:00.589739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06398f2-4fe1-4f76-b0cc-267a95e77f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:00.591179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06398f2-4fe1-4f76-b0cc-267a95e77f49 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:00.601695 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1178/2347] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:32:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:32:00.775877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eaae26-2785-4702-8a02-58dbe21a1842 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:00.777444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a7eaae26-2785-4702-8a02-58dbe21a1842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:00.795492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eaae26-2785-4702-8a02-58dbe21a1842 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:00.800510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a85b77-686c-4d7a-ab11-264ec57f0e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:00.801013 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a85b77-686c-4d7a-ab11-264ec57f0e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:00.801346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a85b77-686c-4d7a-ab11-264ec57f0e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:00.801620 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a85b77-686c-4d7a-ab11-264ec57f0e81 admin admin] PATH_INFO: `/v3/projects/6619450eef5a4a5b8d3b6a45fe412788` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:00.802751 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a85b77-686c-4d7a-ab11-264ec57f0e81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6619450eef5a4a5b8d3b6a45fe412788)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:00.804662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a85b77-686c-4d7a-ab11-264ec57f0e81 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:00.858711 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c1a85b77-686c-4d7a-ab11-264ec57f0e81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6619450eef5a4a5b8d3b6a45fe412788', 'request_id': 'req-c1a85b77-686c-4d7a-ab11-264ec57f0e81'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:00.867336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c1a85b77-686c-4d7a-ab11-264ec57f0e81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6619450eef5a4a5b8d3b6a45fe412788', 'request_id': 'req-c1a85b77-686c-4d7a-ab11-264ec57f0e81'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:00.874946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c1a85b77-686c-4d7a-ab11-264ec57f0e81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6619450eef5a4a5b8d3b6a45fe412788', 'request_id': 'req-c1a85b77-686c-4d7a-ab11-264ec57f0e81'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:00.887951 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c1a85b77-686c-4d7a-ab11-264ec57f0e81 admin admin] The token cache is being invalidate because project 6619450eef5a4a5b8d3b6a45fe412788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:32:00.888307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c1a85b77-686c-4d7a-ab11-264ec57f0e81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:00.889272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c1a85b77-686c-4d7a-ab11-264ec57f0e81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:00.891619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1170/2348] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:32:00 2022] DELETE /identity/v3/projects/6619450eef5a4a5b8d3b6a45fe412788 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:32:00.896585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5b43c-92da-46be-a5fd-f6a670c25242 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:00.896955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5b43c-92da-46be-a5fd-f6a670c25242 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:00.899487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5b43c-92da-46be-a5fd-f6a670c25242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:00.917772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b0a5b43c-92da-46be-a5fd-f6a670c25242 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:32:00.919068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b0a5b43c-92da-46be-a5fd-f6a670c25242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:00.986904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1179/2349] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:32:00.994696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd2a9c-252c-4041-9579-d4e4b1bba5dd None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:00.994992 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd2a9c-252c-4041-9579-d4e4b1bba5dd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:00.995262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd2a9c-252c-4041-9579-d4e4b1bba5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:01.021015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d1cd2a9c-252c-4041-9579-d4e4b1bba5dd None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:32:01.022504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d1cd2a9c-252c-4041-9579-d4e4b1bba5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:01.030886 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1171/2350] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:32:01.035265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5b43c-92da-46be-a5fd-f6a670c25242 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:01.045385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5b43c-92da-46be-a5fd-f6a670c25242 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:01.050659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4114e-b72d-489d-95b2-ff466e2693be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:01.051340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4114e-b72d-489d-95b2-ff466e2693be admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:01.051652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4114e-b72d-489d-95b2-ff466e2693be admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:01.051993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4114e-b72d-489d-95b2-ff466e2693be admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:01.052720 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4114e-b72d-489d-95b2-ff466e2693be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:32:01.053129 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4114e-b72d-489d-95b2-ff466e2693be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:01.054855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4114e-b72d-489d-95b2-ff466e2693be admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:01.066943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1180/2351] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:32:01.071258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd2a9c-252c-4041-9579-d4e4b1bba5dd None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:01.080598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd2a9c-252c-4041-9579-d4e4b1bba5dd None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:01.085484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212480d2-cda2-4077-ba8b-66b72fcd4d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:01.086085 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212480d2-cda2-4077-ba8b-66b72fcd4d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:01.086394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212480d2-cda2-4077-ba8b-66b72fcd4d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:01.086665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212480d2-cda2-4077-ba8b-66b72fcd4d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:01.087331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212480d2-cda2-4077-ba8b-66b72fcd4d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:01.089225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212480d2-cda2-4077-ba8b-66b72fcd4d15 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:01.118248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1172/2352] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:32:01 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:32:01.122562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4114e-b72d-489d-95b2-ff466e2693be admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:01.132647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4114e-b72d-489d-95b2-ff466e2693be admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:01.190761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b144d-059b-4a19-9507-2e97a19e45fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:01.191426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b144d-059b-4a19-9507-2e97a19e45fa admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:01.191636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b144d-059b-4a19-9507-2e97a19e45fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:01.191929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b144d-059b-4a19-9507-2e97a19e45fa admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:01.192548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b144d-059b-4a19-9507-2e97a19e45fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:01.194501 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b144d-059b-4a19-9507-2e97a19e45fa admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:01.219530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1181/2353] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:32:01 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:32:01.223749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212480d2-cda2-4077-ba8b-66b72fcd4d15 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:01.235439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212480d2-cda2-4077-ba8b-66b72fcd4d15 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:01.241455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5a568-e2a6-4924-a120-f6166a94436c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:01.242075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5a568-e2a6-4924-a120-f6166a94436c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:01.242361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5a568-e2a6-4924-a120-f6166a94436c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:01.242628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5a568-e2a6-4924-a120-f6166a94436c admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:01.243780 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5a568-e2a6-4924-a120-f6166a94436c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:01.245814 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5a568-e2a6-4924-a120-f6166a94436c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:01.258131 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1173/2354] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:32:01.261990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b144d-059b-4a19-9507-2e97a19e45fa admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:01.273374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b144d-059b-4a19-9507-2e97a19e45fa admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:01.281319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e1bf1-d72c-4132-8b68-683185a93412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:01.281999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e1bf1-d72c-4132-8b68-683185a93412 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:01.282286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e1bf1-d72c-4132-8b68-683185a93412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:01.282559 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e1bf1-d72c-4132-8b68-683185a93412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:01.283251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e1bf1-d72c-4132-8b68-683185a93412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:01.284887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e1bf1-d72c-4132-8b68-683185a93412 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:01.296034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1182/2355] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:32:01.301635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5a568-e2a6-4924-a120-f6166a94436c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:01.314791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5a568-e2a6-4924-a120-f6166a94436c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:01.319727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49491f3-70ba-4833-bf76-4dad34007eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:01.320394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49491f3-70ba-4833-bf76-4dad34007eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:01.320845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49491f3-70ba-4833-bf76-4dad34007eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:01.321001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49491f3-70ba-4833-bf76-4dad34007eb7 admin admin] PATH_INFO: `/v3/projects/9d47a53afdbf404a829c119992fcb643/users/166d99888375478e9cd8bf8dcb9d4190/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:01.346679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49491f3-70ba-4833-bf76-4dad34007eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d47a53afdbf404a829c119992fcb643, user_id=166d99888375478e9cd8bf8dcb9d4190, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:01.351377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49491f3-70ba-4833-bf76-4dad34007eb7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:01.371923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1174/2356] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:32:01 2022] PUT /identity/v3/projects/9d47a53afdbf404a829c119992fcb643/users/166d99888375478e9cd8bf8dcb9d4190/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:32:01.378702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e82dc-379f-4c33-a839-6a447fbf7a4d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:01.378990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e82dc-379f-4c33-a839-6a447fbf7a4d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:01.379262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e82dc-379f-4c33-a839-6a447fbf7a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:01.399176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-501e82dc-379f-4c33-a839-6a447fbf7a4d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:32:01.400449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-501e82dc-379f-4c33-a839-6a447fbf7a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:01.459472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:01.484138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1183/2357] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:32:01.494084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49491f3-70ba-4833-bf76-4dad34007eb7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:01.495803 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c49491f3-70ba-4833-bf76-4dad34007eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:01.551248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49491f3-70ba-4833-bf76-4dad34007eb7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:01.555058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28977223-2d8e-43d2-aade-6c9a36a95d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:01.555662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28977223-2d8e-43d2-aade-6c9a36a95d12 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:01.555939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28977223-2d8e-43d2-aade-6c9a36a95d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:01.556204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28977223-2d8e-43d2-aade-6c9a36a95d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:01.565215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28977223-2d8e-43d2-aade-6c9a36a95d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:01.566622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28977223-2d8e-43d2-aade-6c9a36a95d12 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:01.575971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1175/2358] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:32:02.221552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb744f9-6779-419c-aa4d-e38b1aed4164 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:02.222063 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb744f9-6779-419c-aa4d-e38b1aed4164 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:02.222159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb744f9-6779-419c-aa4d-e38b1aed4164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:02.252617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3fb744f9-6779-419c-aa4d-e38b1aed4164 None None] MFA Rules not processed for user `166d99888375478e9cd8bf8dcb9d4190`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:32:02.254025 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3fb744f9-6779-419c-aa4d-e38b1aed4164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:02.325556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1184/2359] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:32:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:32:02.336404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28977223-2d8e-43d2-aade-6c9a36a95d12 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:02.337959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-28977223-2d8e-43d2-aade-6c9a36a95d12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:02.343515 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:02.399257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28977223-2d8e-43d2-aade-6c9a36a95d12 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:02.402992 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc79687-a4c1-40ed-b189-5688631f370e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:02.403606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc79687-a4c1-40ed-b189-5688631f370e service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:02.403881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc79687-a4c1-40ed-b189-5688631f370e service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:02.404149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc79687-a4c1-40ed-b189-5688631f370e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:02.412706 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc79687-a4c1-40ed-b189-5688631f370e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:02.414167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc79687-a4c1-40ed-b189-5688631f370e service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:02.426804 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1176/2360] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:32:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:32:08.064150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605914de-1502-4d67-9389-5eaf8015b229 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:08.064682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605914de-1502-4d67-9389-5eaf8015b229 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:08.064682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605914de-1502-4d67-9389-5eaf8015b229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:08.085760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-605914de-1502-4d67-9389-5eaf8015b229 None None] MFA Rules not processed for user `166d99888375478e9cd8bf8dcb9d4190`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:32:08.087256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-605914de-1502-4d67-9389-5eaf8015b229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:08.094309 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1185/2361] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:32:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:32:10.007044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc79687-a4c1-40ed-b189-5688631f370e service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:10.008746 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7fc79687-a4c1-40ed-b189-5688631f370e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:10.023145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc79687-a4c1-40ed-b189-5688631f370e service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:10.027902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:10.028514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:10.028778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:10.029047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] PATH_INFO: `/v3/users/1195b16983ca4363abc1cd2829ecdee8` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:10.030212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1195b16983ca4363abc1cd2829ecdee8)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:10.032089 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:10.081676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1195b16983ca4363abc1cd2829ecdee8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:10.089700 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1195b16983ca4363abc1cd2829ecdee8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:10.090340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1195b16983ca4363abc1cd2829ecdee8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:10.097416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1195b16983ca4363abc1cd2829ecdee8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:10.110220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1195b16983ca4363abc1cd2829ecdee8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:10.116335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1177/2362] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:32:10 2022] DELETE /identity/v3/users/1195b16983ca4363abc1cd2829ecdee8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:32:10.120933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276b877-5e99-47c5-a390-0c6326e959ef None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:10.121470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276b877-5e99-47c5-a390-0c6326e959ef None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:10.121644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276b877-5e99-47c5-a390-0c6326e959ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:10.140576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b276b877-5e99-47c5-a390-0c6326e959ef None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:32:10.141818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b276b877-5e99-47c5-a390-0c6326e959ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:10.206980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1186/2363] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:32:10.216749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:10.218358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:10.292249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff79b02-9668-4fe7-81bf-685d6740d43a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:10.296728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb52d04-44c1-4ae8-bc59-fbce921b6dd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:10.297380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb52d04-44c1-4ae8-bc59-fbce921b6dd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:10.297658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb52d04-44c1-4ae8-bc59-fbce921b6dd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:10.297928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb52d04-44c1-4ae8-bc59-fbce921b6dd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:10.306829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb52d04-44c1-4ae8-bc59-fbce921b6dd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:10.308253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb52d04-44c1-4ae8-bc59-fbce921b6dd7 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:10.317748 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1178/2364] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:32:10.719487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276b877-5e99-47c5-a390-0c6326e959ef None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:10.722076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b276b877-5e99-47c5-a390-0c6326e959ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:10.734861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276b877-5e99-47c5-a390-0c6326e959ef None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:10.740206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:10.741218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:10.741706 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:10.741945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a admin admin] PATH_INFO: `/v3/projects/35e627b87be74f679c84863eb37f9a6f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:10.743164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e627b87be74f679c84863eb37f9a6f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:10.745243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:10.794342 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e627b87be74f679c84863eb37f9a6f', 'request_id': 'req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:10.801945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e627b87be74f679c84863eb37f9a6f', 'request_id': 'req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:10.807004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:10.808341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e627b87be74f679c84863eb37f9a6f', 'request_id': 'req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:10.822604 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a admin admin] The token cache is being invalidate because project 35e627b87be74f679c84863eb37f9a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:32:10.822893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:10.823578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-51b13396-cb0b-48ba-b6a5-a8f6bcade56a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:32:10.825098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1187/2365] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:32:10 2022] DELETE /identity/v3/projects/35e627b87be74f679c84863eb37f9a6f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:32:14.487125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb52d04-44c1-4ae8-bc59-fbce921b6dd7 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:14.488554 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6cb52d04-44c1-4ae8-bc59-fbce921b6dd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:14.500082 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb52d04-44c1-4ae8-bc59-fbce921b6dd7 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:14.516816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:14.550282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf5dfb-b9cb-45f3-8ffb-91074d21efda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:14.550880 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf5dfb-b9cb-45f3-8ffb-91074d21efda service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:14.551148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf5dfb-b9cb-45f3-8ffb-91074d21efda service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:14.551416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf5dfb-b9cb-45f3-8ffb-91074d21efda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:14.553319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b6cf5dfb-b9cb-45f3-8ffb-91074d21efda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:14.628971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf5dfb-b9cb-45f3-8ffb-91074d21efda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:14.630465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf5dfb-b9cb-45f3-8ffb-91074d21efda service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:14.643260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1179/2366] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:32:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:32:20.819503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:24.637422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=41 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:29.645420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3f5d9-0d1a-4647-934b-703f476e5c95 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:29.645935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3f5d9-0d1a-4647-934b-703f476e5c95 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:29.645935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3f5d9-0d1a-4647-934b-703f476e5c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:29.666051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-87d3f5d9-0d1a-4647-934b-703f476e5c95 None None] MFA Rules not processed for user `094abcdcd1f54a658564357056e7849a`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:32:29.667223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-87d3f5d9-0d1a-4647-934b-703f476e5c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:29.725104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1188/2367] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:32:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:32:35.710852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464cda2-012e-444a-92ef-09f93b0b835b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:35.711715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464cda2-012e-444a-92ef-09f93b0b835b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:35.711991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464cda2-012e-444a-92ef-09f93b0b835b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:35.732926 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d464cda2-012e-444a-92ef-09f93b0b835b None None] MFA Rules not processed for user `5bb515269d1a45f594f3d297c7fd267e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:32:35.734239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d464cda2-012e-444a-92ef-09f93b0b835b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:35.790876 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1180/2368] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:32:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:32:39.717947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=25 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:45.783566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=25 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:59.325739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b80b1-2ad4-410e-92e2-b628bcdb2590 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:59.326329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b80b1-2ad4-410e-92e2-b628bcdb2590 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:59.326329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b80b1-2ad4-410e-92e2-b628bcdb2590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:59.347700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-440b80b1-2ad4-410e-92e2-b628bcdb2590 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:32:59.348914 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-440b80b1-2ad4-410e-92e2-b628bcdb2590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:59.430624 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1189/2369] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:32:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:32:59.489671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464cda2-012e-444a-92ef-09f93b0b835b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:32:59.492242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d464cda2-012e-444a-92ef-09f93b0b835b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:32:59.587529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464cda2-012e-444a-92ef-09f93b0b835b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:32:59.598892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b295c45-d276-4d2c-a542-29a105cb2dcc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:32:59.599709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b295c45-d276-4d2c-a542-29a105cb2dcc service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:32:59.600091 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b295c45-d276-4d2c-a542-29a105cb2dcc service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:32:59.600499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b295c45-d276-4d2c-a542-29a105cb2dcc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:32:59.628251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b295c45-d276-4d2c-a542-29a105cb2dcc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:32:59.630496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b295c45-d276-4d2c-a542-29a105cb2dcc service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:32:59.678419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1181/2370] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:32:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:33:09.421925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=25 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:09.651658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=32 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:21.536672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b80b1-2ad4-410e-92e2-b628bcdb2590 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:33:21.538165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-440b80b1-2ad4-410e-92e2-b628bcdb2590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:33:21.549643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b80b1-2ad4-410e-92e2-b628bcdb2590 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:33:21.553375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f143d39-f55d-4ca7-b6a0-c01ca59e7bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:33:21.553982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f143d39-f55d-4ca7-b6a0-c01ca59e7bdf service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:21.554260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f143d39-f55d-4ca7-b6a0-c01ca59e7bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:21.554522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f143d39-f55d-4ca7-b6a0-c01ca59e7bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:21.556419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8f143d39-f55d-4ca7-b6a0-c01ca59e7bdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:33:21.629722 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f143d39-f55d-4ca7-b6a0-c01ca59e7bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:33:21.631150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f143d39-f55d-4ca7-b6a0-c01ca59e7bdf service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:33:21.642580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1190/2371] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:33:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:33:22.656137 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b295c45-d276-4d2c-a542-29a105cb2dcc service swift] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:33:22.657673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8b295c45-d276-4d2c-a542-29a105cb2dcc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:33:22.673289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b295c45-d276-4d2c-a542-29a105cb2dcc service swift] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:33:22.677409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9314a9-b4bb-4bf1-a316-55fd6857a50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:33:22.678004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9314a9-b4bb-4bf1-a316-55fd6857a50f service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:22.678394 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9314a9-b4bb-4bf1-a316-55fd6857a50f service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:22.678748 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9314a9-b4bb-4bf1-a316-55fd6857a50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:22.680603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9c9314a9-b4bb-4bf1-a316-55fd6857a50f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:33:22.695065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9314a9-b4bb-4bf1-a316-55fd6857a50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:33:22.696502 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9314a9-b4bb-4bf1-a316-55fd6857a50f service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:33:22.708177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1182/2372] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:33:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:33:23.494123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1b88b-6b24-41d0-94c6-d3be9347e1a1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:23.494673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1b88b-6b24-41d0-94c6-d3be9347e1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:23.494804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1b88b-6b24-41d0-94c6-d3be9347e1a1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:23.496097 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1191/2373] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:33:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:33:23.502586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7171b0-6144-47bc-8840-f4c4ebc0a7f1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:23.502870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7171b0-6144-47bc-8840-f4c4ebc0a7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:23.503138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7171b0-6144-47bc-8840-f4c4ebc0a7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:23.526274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5c7171b0-6144-47bc-8840-f4c4ebc0a7f1 None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:33:23.527482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5c7171b0-6144-47bc-8840-f4c4ebc0a7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:33:23.540728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1183/2374] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:33:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:33:23.553337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1b88b-6b24-41d0-94c6-d3be9347e1a1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:33:23.555134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-12a1b88b-6b24-41d0-94c6-d3be9347e1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:33:23.617038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1b88b-6b24-41d0-94c6-d3be9347e1a1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:33:23.620859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bfebb-bc93-48e3-a887-0e678a22edcf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf8a0ae03c94aa78cc2ab0e4e1c87b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:33:23.621543 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bfebb-bc93-48e3-a887-0e678a22edcf service glance] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:23.622006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bfebb-bc93-48e3-a887-0e678a22edcf service glance] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:23.622282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bfebb-bc93-48e3-a887-0e678a22edcf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:23.632578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bfebb-bc93-48e3-a887-0e678a22edcf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:33:23.634072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bfebb-bc93-48e3-a887-0e678a22edcf service glance] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:33:23.644878 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1192/2375] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:33:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:33:23.715292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f767a0-a17e-4ba9-b376-481205dfe6ff None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:23.715877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f767a0-a17e-4ba9-b376-481205dfe6ff None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:23.715877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f767a0-a17e-4ba9-b376-481205dfe6ff None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:23.717884 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1184/2376] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:33:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:33:23.722961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1b559-2ed9-4180-80d8-b012626e8977 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:23.723247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1b559-2ed9-4180-80d8-b012626e8977 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:23.723519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1b559-2ed9-4180-80d8-b012626e8977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:23.744518 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b5f1b559-2ed9-4180-80d8-b012626e8977 None None] MFA Rules not processed for user `c9360be7ee9e4739b20c59955d28da47`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:33:23.745767 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b5f1b559-2ed9-4180-80d8-b012626e8977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:33:23.752665 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1193/2377] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:33:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:33:23.764761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f767a0-a17e-4ba9-b376-481205dfe6ff None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:33:23.777262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f767a0-a17e-4ba9-b376-481205dfe6ff None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:33:23.781108 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde0f11-186d-4726-ac84-41eb587af039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:33:23.781720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde0f11-186d-4726-ac84-41eb587af039 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:23.781994 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde0f11-186d-4726-ac84-41eb587af039 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:23.782261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde0f11-186d-4726-ac84-41eb587af039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:23.789628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde0f11-186d-4726-ac84-41eb587af039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:33:23.791046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde0f11-186d-4726-ac84-41eb587af039 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:33:23.798698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1185/2378] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:33:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:33:23.885736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026e48e-03bc-4e6f-be80-d4fab3827ee5 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:23.886066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026e48e-03bc-4e6f-be80-d4fab3827ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:23.886539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026e48e-03bc-4e6f-be80-d4fab3827ee5 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:23.889569 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1194/2379] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:33:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:33:23.895052 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93aee0a-85b9-450f-8a71-ecb5c229d432 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:23.895315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93aee0a-85b9-450f-8a71-ecb5c229d432 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:23.895607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93aee0a-85b9-450f-8a71-ecb5c229d432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:23.917973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a93aee0a-85b9-450f-8a71-ecb5c229d432 None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:33:23.919587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a93aee0a-85b9-450f-8a71-ecb5c229d432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:33:23.929020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1186/2380] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:33:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:33:23.940303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026e48e-03bc-4e6f-be80-d4fab3827ee5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:33:23.952498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026e48e-03bc-4e6f-be80-d4fab3827ee5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:33:23.956250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde2eb1-b0b0-4cbb-ba6d-3256be05b355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:33:23.956853 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde2eb1-b0b0-4cbb-ba6d-3256be05b355 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:23.957158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde2eb1-b0b0-4cbb-ba6d-3256be05b355 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:23.957432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde2eb1-b0b0-4cbb-ba6d-3256be05b355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:23.967882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde2eb1-b0b0-4cbb-ba6d-3256be05b355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:33:23.969352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde2eb1-b0b0-4cbb-ba6d-3256be05b355 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:33:23.980474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1195/2381] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:33:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:33:30.068461 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93aee0a-85b9-450f-8a71-ecb5c229d432 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:33:30.079606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93aee0a-85b9-450f-8a71-ecb5c229d432 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:33:30.084295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c9b75-c7da-4af7-a72f-8c9726625def service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf8a0ae03c94aa78cc2ab0e4e1c87b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:33:30.084922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c9b75-c7da-4af7-a72f-8c9726625def service glance] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:30.085233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c9b75-c7da-4af7-a72f-8c9726625def service glance] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:30.085507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c9b75-c7da-4af7-a72f-8c9726625def service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:30.087589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-dd1c9b75-c7da-4af7-a72f-8c9726625def service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:33:30.153420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c9b75-c7da-4af7-a72f-8c9726625def service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:33:30.154874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c9b75-c7da-4af7-a72f-8c9726625def service glance] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:33:30.163875 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1187/2382] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:33:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:33:30.366420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7c4d0-a428-4cc0-9a7c-26cf143f069f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:30.366943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7c4d0-a428-4cc0-9a7c-26cf143f069f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:30.366943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7c4d0-a428-4cc0-9a7c-26cf143f069f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:30.385767 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-62a7c4d0-a428-4cc0-9a7c-26cf143f069f None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:33:30.386960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-62a7c4d0-a428-4cc0-9a7c-26cf143f069f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:33:30.394858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1196/2383] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:33:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:33:30.408082 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c9b75-c7da-4af7-a72f-8c9726625def service glance] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:33:30.420040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c9b75-c7da-4af7-a72f-8c9726625def service glance] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:33:30.426517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e508b5-57cf-47d4-bce3-45125257562a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:33:30.427308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e508b5-57cf-47d4-bce3-45125257562a service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:30.427627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e508b5-57cf-47d4-bce3-45125257562a service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:30.427951 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e508b5-57cf-47d4-bce3-45125257562a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:30.437752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e508b5-57cf-47d4-bce3-45125257562a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:33:30.439401 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e508b5-57cf-47d4-bce3-45125257562a service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:33:30.448611 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1188/2384] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:33:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:33:40.380806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=72 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:40.445211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=52 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:47.596685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7c4d0-a428-4cc0-9a7c-26cf143f069f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:33:47.606994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7c4d0-a428-4cc0-9a7c-26cf143f069f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:33:47.611033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa837e3-fc9e-49e4-bbac-d5ad5f04bf95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:33:47.611679 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa837e3-fc9e-49e4-bbac-d5ad5f04bf95 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:33:47.611976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa837e3-fc9e-49e4-bbac-d5ad5f04bf95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:33:47.612242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa837e3-fc9e-49e4-bbac-d5ad5f04bf95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:33:47.614131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-baa837e3-fc9e-49e4-bbac-d5ad5f04bf95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:33:47.624174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa837e3-fc9e-49e4-bbac-d5ad5f04bf95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:33:47.626114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa837e3-fc9e-49e4-bbac-d5ad5f04bf95 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:33:47.634866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1197/2385] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:33:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:33:57.630690 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:34:00.965247 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e6a27-898e-4374-a8e6-fa14e9b607d6 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:34:00.965839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e6a27-898e-4374-a8e6-fa14e9b607d6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:34:00.965839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e6a27-898e-4374-a8e6-fa14e9b607d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:34:00.985769 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-da5e6a27-898e-4374-a8e6-fa14e9b607d6 None None] MFA Rules not processed for user `166d99888375478e9cd8bf8dcb9d4190`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:34:00.986979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-da5e6a27-898e-4374-a8e6-fa14e9b607d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:34:01.047834 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1189/2386] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:34:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:34:01.300122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa837e3-fc9e-49e4-bbac-d5ad5f04bf95 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:34:01.309372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa837e3-fc9e-49e4-bbac-d5ad5f04bf95 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:34:01.313361 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad0f9d-f622-4e33-9552-70b44f26de8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:34:01.313968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad0f9d-f622-4e33-9552-70b44f26de8c service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:34:01.314245 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad0f9d-f622-4e33-9552-70b44f26de8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:34:01.314512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad0f9d-f622-4e33-9552-70b44f26de8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:34:01.316374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2fad0f9d-f622-4e33-9552-70b44f26de8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:34:01.326345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad0f9d-f622-4e33-9552-70b44f26de8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:34:01.327806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad0f9d-f622-4e33-9552-70b44f26de8c service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:34:01.336099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1198/2387] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:34:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:34:01.661622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e6a27-898e-4374-a8e6-fa14e9b607d6 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:34:01.663230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-da5e6a27-898e-4374-a8e6-fa14e9b607d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:34:01.736765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e6a27-898e-4374-a8e6-fa14e9b607d6 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:34:01.742196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42428f-dac9-4afc-99d3-fc1165d3c27d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:34:01.743042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42428f-dac9-4afc-99d3-fc1165d3c27d service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:34:01.743345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42428f-dac9-4afc-99d3-fc1165d3c27d service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:34:01.743621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42428f-dac9-4afc-99d3-fc1165d3c27d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:34:01.745427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7b42428f-dac9-4afc-99d3-fc1165d3c27d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:34:01.757114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42428f-dac9-4afc-99d3-fc1165d3c27d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:34:01.758710 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42428f-dac9-4afc-99d3-fc1165d3c27d service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:34:01.768666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1190/2388] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:34:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:34:05.987788 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129187e-fe54-4092-b335-1e01a2012ea4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:34:05.988279 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129187e-fe54-4092-b335-1e01a2012ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:34:05.988372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129187e-fe54-4092-b335-1e01a2012ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:34:06.008369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f129187e-fe54-4092-b335-1e01a2012ea4 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:34:06.009617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f129187e-fe54-4092-b335-1e01a2012ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:34:06.017814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1199/2389] 10.222.0.60 () {58 vars in 967 bytes} [Sat Sep 17 04:34:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:34:06.041107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42428f-dac9-4afc-99d3-fc1165d3c27d service placement] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:34:06.060989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42428f-dac9-4afc-99d3-fc1165d3c27d service placement] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:34:06.066528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef3f09-ba7d-48aa-9c99-818be113ae12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:34:06.067126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef3f09-ba7d-48aa-9c99-818be113ae12 service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:34:06.067403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef3f09-ba7d-48aa-9c99-818be113ae12 service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:34:06.067865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef3f09-ba7d-48aa-9c99-818be113ae12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:34:06.079210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef3f09-ba7d-48aa-9c99-818be113ae12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:34:06.081215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef3f09-ba7d-48aa-9c99-818be113ae12 service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:34:06.092089 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1191/2390] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:34:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:34:07.341322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129187e-fe54-4092-b335-1e01a2012ea4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:34:07.343936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f129187e-fe54-4092-b335-1e01a2012ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:34:07.362655 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129187e-fe54-4092-b335-1e01a2012ea4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:34:07.370241 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65679cd8-2b2c-4240-871b-098b3b77ade6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:34:07.371254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65679cd8-2b2c-4240-871b-098b3b77ade6 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:34:07.371673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65679cd8-2b2c-4240-871b-098b3b77ade6 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:34:07.372210 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65679cd8-2b2c-4240-871b-098b3b77ade6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:34:07.374902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-65679cd8-2b2c-4240-871b-098b3b77ade6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:34:07.388594 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65679cd8-2b2c-4240-871b-098b3b77ade6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:34:07.390488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65679cd8-2b2c-4240-871b-098b3b77ade6 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:34:07.401948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1200/2391] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:34:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:34:14.884424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef3f09-ba7d-48aa-9c99-818be113ae12 service swift] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:34:14.886096 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a5ef3f09-ba7d-48aa-9c99-818be113ae12 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:34:14.902037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef3f09-ba7d-48aa-9c99-818be113ae12 service swift] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:34:14.908100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98009f89-cba8-4a0a-b359-3378de3c6a32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:34:14.908712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98009f89-cba8-4a0a-b359-3378de3c6a32 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:34:14.908991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98009f89-cba8-4a0a-b359-3378de3c6a32 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:34:14.909304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98009f89-cba8-4a0a-b359-3378de3c6a32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:34:14.911334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-98009f89-cba8-4a0a-b359-3378de3c6a32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:34:14.923862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98009f89-cba8-4a0a-b359-3378de3c6a32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:34:14.925527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98009f89-cba8-4a0a-b359-3378de3c6a32 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:34:14.936554 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1192/2392] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:34:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:34:17.397603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=15 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:34:24.931961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=69 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:03.481492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65679cd8-2b2c-4240-871b-098b3b77ade6 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:03.499617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65679cd8-2b2c-4240-871b-098b3b77ade6 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:03.507973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68124efe-47cf-44fc-926e-a1e1947ecbb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:03.508570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68124efe-47cf-44fc-926e-a1e1947ecbb6 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:03.508847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68124efe-47cf-44fc-926e-a1e1947ecbb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:03.509149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68124efe-47cf-44fc-926e-a1e1947ecbb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:03.511272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-68124efe-47cf-44fc-926e-a1e1947ecbb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:03.523624 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68124efe-47cf-44fc-926e-a1e1947ecbb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:03.525859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68124efe-47cf-44fc-926e-a1e1947ecbb6 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:03.539657 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1201/2393] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:35:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:35:06.902923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98009f89-cba8-4a0a-b359-3378de3c6a32 service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:06.917243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98009f89-cba8-4a0a-b359-3378de3c6a32 service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:06.923837 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca6595-8b3b-4221-b8ac-407ab128e6ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:06.924638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca6595-8b3b-4221-b8ac-407ab128e6ba service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:06.925210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca6595-8b3b-4221-b8ac-407ab128e6ba service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:06.925564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca6595-8b3b-4221-b8ac-407ab128e6ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:06.928963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5aca6595-8b3b-4221-b8ac-407ab128e6ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:06.991686 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca6595-8b3b-4221-b8ac-407ab128e6ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:06.993311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca6595-8b3b-4221-b8ac-407ab128e6ba service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:07.002426 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1193/2394] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:35:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:35:07.732333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68124efe-47cf-44fc-926e-a1e1947ecbb6 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:07.743439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68124efe-47cf-44fc-926e-a1e1947ecbb6 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:07.749170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e803a3-125a-4ca2-88ad-da538547d4a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:07.750135 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e803a3-125a-4ca2-88ad-da538547d4a4 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:07.750394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e803a3-125a-4ca2-88ad-da538547d4a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:07.750818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e803a3-125a-4ca2-88ad-da538547d4a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:07.752947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-34e803a3-125a-4ca2-88ad-da538547d4a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:07.766820 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e803a3-125a-4ca2-88ad-da538547d4a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:07.768792 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e803a3-125a-4ca2-88ad-da538547d4a4 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:07.781155 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1202/2395] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:35:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:35:13.959786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca6595-8b3b-4221-b8ac-407ab128e6ba service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:13.961420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5aca6595-8b3b-4221-b8ac-407ab128e6ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:14.022259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca6595-8b3b-4221-b8ac-407ab128e6ba service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:14.026137 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbab30-c14a-4d00-bb02-38e0a3f708d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:14.026695 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbab30-c14a-4d00-bb02-38e0a3f708d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:14.026976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbab30-c14a-4d00-bb02-38e0a3f708d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:14.027238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbab30-c14a-4d00-bb02-38e0a3f708d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:14.029115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-65cbab30-c14a-4d00-bb02-38e0a3f708d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:14.040942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbab30-c14a-4d00-bb02-38e0a3f708d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:14.042414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbab30-c14a-4d00-bb02-38e0a3f708d3 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:14.052777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1194/2396] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:35:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:35:17.036936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e803a3-125a-4ca2-88ad-da538547d4a4 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:17.049275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e803a3-125a-4ca2-88ad-da538547d4a4 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:17.054743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda407-ba9b-48f7-a5ae-ee1e597c5741 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:17.055726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda407-ba9b-48f7-a5ae-ee1e597c5741 service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:17.055969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda407-ba9b-48f7-a5ae-ee1e597c5741 service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:17.056317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda407-ba9b-48f7-a5ae-ee1e597c5741 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:17.058171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-70cda407-ba9b-48f7-a5ae-ee1e597c5741 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:17.069969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda407-ba9b-48f7-a5ae-ee1e597c5741 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:17.071624 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda407-ba9b-48f7-a5ae-ee1e597c5741 service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:17.082481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1203/2397] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:35:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:35:17.091115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbab30-c14a-4d00-bb02-38e0a3f708d3 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:17.092734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-65cbab30-c14a-4d00-bb02-38e0a3f708d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:17.160759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbab30-c14a-4d00-bb02-38e0a3f708d3 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:17.165469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd90b4-ab19-4913-96cb-22d36b5f0f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:17.166098 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd90b4-ab19-4913-96cb-22d36b5f0f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:17.166371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd90b4-ab19-4913-96cb-22d36b5f0f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:17.166638 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd90b4-ab19-4913-96cb-22d36b5f0f79 admin admin] PATH_INFO: `/v3/users/166d99888375478e9cd8bf8dcb9d4190` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:17.167807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd90b4-ab19-4913-96cb-22d36b5f0f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=166d99888375478e9cd8bf8dcb9d4190)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:17.169832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd90b4-ab19-4913-96cb-22d36b5f0f79 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:17.217839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:17.221333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6dcd90b4-ab19-4913-96cb-22d36b5f0f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '166d99888375478e9cd8bf8dcb9d4190'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:17.232432 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6dcd90b4-ab19-4913-96cb-22d36b5f0f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '166d99888375478e9cd8bf8dcb9d4190'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:17.233068 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6dcd90b4-ab19-4913-96cb-22d36b5f0f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '166d99888375478e9cd8bf8dcb9d4190'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:17.256591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6dcd90b4-ab19-4913-96cb-22d36b5f0f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '166d99888375478e9cd8bf8dcb9d4190'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:17.299163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6dcd90b4-ab19-4913-96cb-22d36b5f0f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '166d99888375478e9cd8bf8dcb9d4190'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:17.310659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1195/2398] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:35:17 2022] DELETE /identity/v3/users/166d99888375478e9cd8bf8dcb9d4190 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:35:17.702544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda407-ba9b-48f7-a5ae-ee1e597c5741 service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:17.738740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda407-ba9b-48f7-a5ae-ee1e597c5741 service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:17.853729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:17.854487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:17.854774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:17.855382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] PATH_INFO: `/v3/projects/9d47a53afdbf404a829c119992fcb643` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:17.856692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d47a53afdbf404a829c119992fcb643)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:17.858762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:17.943574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d47a53afdbf404a829c119992fcb643', 'request_id': 'req-badde0bf-b943-4be7-8b67-1d2847ac0f82'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:17.969208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d47a53afdbf404a829c119992fcb643', 'request_id': 'req-badde0bf-b943-4be7-8b67-1d2847ac0f82'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:17.985430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d47a53afdbf404a829c119992fcb643', 'request_id': 'req-badde0bf-b943-4be7-8b67-1d2847ac0f82'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:18.000958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] The token cache is being invalidate because project 9d47a53afdbf404a829c119992fcb643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:35:18.001303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:18.002952 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:18.003876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1204/2399] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:35:17 2022] DELETE /identity/v3/projects/9d47a53afdbf404a829c119992fcb643 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:35:19.771110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15531c-84de-4061-8683-88b5a2cb644f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:19.771654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15531c-84de-4061-8683-88b5a2cb644f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:19.771762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15531c-84de-4061-8683-88b5a2cb644f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:19.791121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6b15531c-84de-4061-8683-88b5a2cb644f None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:35:19.792309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6b15531c-84de-4061-8683-88b5a2cb644f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:19.852914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1196/2400] 10.222.0.60 () {58 vars in 967 bytes} [Sat Sep 17 04:35:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:35:19.901701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:19.903165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:19.955063 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:19.975623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badde0bf-b943-4be7-8b67-1d2847ac0f82 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:19.981203 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23a60d-1dc0-46c0-82a1-4a2f0e9248e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:19.981815 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23a60d-1dc0-46c0-82a1-4a2f0e9248e4 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:19.982088 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23a60d-1dc0-46c0-82a1-4a2f0e9248e4 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:19.982352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23a60d-1dc0-46c0-82a1-4a2f0e9248e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:19.990516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23a60d-1dc0-46c0-82a1-4a2f0e9248e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:19.992087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23a60d-1dc0-46c0-82a1-4a2f0e9248e4 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:20.000223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1205/2401] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:35:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:35:20.370153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15531c-84de-4061-8683-88b5a2cb644f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:20.371721 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6b15531c-84de-4061-8683-88b5a2cb644f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:20.433749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15531c-84de-4061-8683-88b5a2cb644f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:20.438588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828017f-3141-4d89-ba7e-4493091ca2b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:20.439331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828017f-3141-4d89-ba7e-4493091ca2b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:20.439627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828017f-3141-4d89-ba7e-4493091ca2b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:20.439898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828017f-3141-4d89-ba7e-4493091ca2b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:20.441725 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d828017f-3141-4d89-ba7e-4493091ca2b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:20.506589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828017f-3141-4d89-ba7e-4493091ca2b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:20.508042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828017f-3141-4d89-ba7e-4493091ca2b5 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:20.517643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1197/2402] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:35:23.307590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babc8d8-935d-422e-8c27-ecd038b98cf4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:23.308100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babc8d8-935d-422e-8c27-ecd038b98cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:23.308200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babc8d8-935d-422e-8c27-ecd038b98cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:23.331391 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1babc8d8-935d-422e-8c27-ecd038b98cf4 None None] MFA Rules not processed for user `094abcdcd1f54a658564357056e7849a`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:35:23.332547 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1babc8d8-935d-422e-8c27-ecd038b98cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:23.392267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1206/2403] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:35:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:35:28.255118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828017f-3141-4d89-ba7e-4493091ca2b5 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:28.256976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d828017f-3141-4d89-ba7e-4493091ca2b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:28.275776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828017f-3141-4d89-ba7e-4493091ca2b5 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:28.281219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:28.281942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:28.282287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:28.282618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] PATH_INFO: `/v3/users/f6a5aff543234d7a9976c2e43453aef2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:28.284123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a5aff543234d7a9976c2e43453aef2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:28.286267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:28.302769 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:28.342867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a5aff543234d7a9976c2e43453aef2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:28.352555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a5aff543234d7a9976c2e43453aef2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:28.354063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a5aff543234d7a9976c2e43453aef2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:28.362008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a5aff543234d7a9976c2e43453aef2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:28.375877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a5aff543234d7a9976c2e43453aef2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:28.382281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1198/2404] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:35:28 2022] DELETE /identity/v3/users/f6a5aff543234d7a9976c2e43453aef2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:35:28.668605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babc8d8-935d-422e-8c27-ecd038b98cf4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:28.707384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babc8d8-935d-422e-8c27-ecd038b98cf4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:28.806534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:28.807444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:28.807872 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:28.808314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] PATH_INFO: `/v3/projects/4ca16cc178624419954df6bad327f722` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:28.810104 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ca16cc178624419954df6bad327f722)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:28.817358 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:28.894026 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ca16cc178624419954df6bad327f722', 'request_id': 'req-02bf2418-be5e-432e-9ca7-0412c1d84072'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:28.905307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ca16cc178624419954df6bad327f722', 'request_id': 'req-02bf2418-be5e-432e-9ca7-0412c1d84072'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:28.912308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ca16cc178624419954df6bad327f722', 'request_id': 'req-02bf2418-be5e-432e-9ca7-0412c1d84072'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:28.934410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] The token cache is being invalidate because project 4ca16cc178624419954df6bad327f722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:35:28.935005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:28.935724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:28.937430 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1207/2405] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:35:28 2022] DELETE /identity/v3/projects/4ca16cc178624419954df6bad327f722 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:35:28.941572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:28.943095 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:28.958939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136b222-a702-4ab7-b659-adda2b2cf297 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:29.037118 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e210c9-15bb-468e-a5d9-4586595b5374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:29.038307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e210c9-15bb-468e-a5d9-4586595b5374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:29.038536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e210c9-15bb-468e-a5d9-4586595b5374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:29.039064 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e210c9-15bb-468e-a5d9-4586595b5374 admin admin] PATH_INFO: `/v3/users/8606d928811c4c64aea5db0c71a81c04` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:29.040585 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e210c9-15bb-468e-a5d9-4586595b5374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8606d928811c4c64aea5db0c71a81c04)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:29.043036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e210c9-15bb-468e-a5d9-4586595b5374 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:29.093383 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-94e210c9-15bb-468e-a5d9-4586595b5374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8606d928811c4c64aea5db0c71a81c04'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:29.103982 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-94e210c9-15bb-468e-a5d9-4586595b5374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8606d928811c4c64aea5db0c71a81c04'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:29.104791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-94e210c9-15bb-468e-a5d9-4586595b5374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8606d928811c4c64aea5db0c71a81c04'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:29.112870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-94e210c9-15bb-468e-a5d9-4586595b5374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8606d928811c4c64aea5db0c71a81c04'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:29.125119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-94e210c9-15bb-468e-a5d9-4586595b5374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8606d928811c4c64aea5db0c71a81c04'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:29.131153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1199/2406] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:35:28 2022] DELETE /identity/v3/users/8606d928811c4c64aea5db0c71a81c04 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:35:29.388855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:29.390664 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:29.404330 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf2418-be5e-432e-9ca7-0412c1d84072 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:29.420244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:29.485873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814892c-b61f-4c01-915c-2ed9d421e4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:29.486421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814892c-b61f-4c01-915c-2ed9d421e4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:29.486693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814892c-b61f-4c01-915c-2ed9d421e4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:29.486958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814892c-b61f-4c01-915c-2ed9d421e4e2 admin admin] PATH_INFO: `/v3/projects/191e745d2d9245da8d2df2e4f9a7814a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:29.488074 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814892c-b61f-4c01-915c-2ed9d421e4e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=191e745d2d9245da8d2df2e4f9a7814a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:29.490021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814892c-b61f-4c01-915c-2ed9d421e4e2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:29.549218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a814892c-b61f-4c01-915c-2ed9d421e4e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '191e745d2d9245da8d2df2e4f9a7814a', 'request_id': 'req-a814892c-b61f-4c01-915c-2ed9d421e4e2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:29.559932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2e4cf-17eb-456b-80ea-bef557d8f320 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:29.560518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2e4cf-17eb-456b-80ea-bef557d8f320 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:29.560685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2e4cf-17eb-456b-80ea-bef557d8f320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:29.564351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a814892c-b61f-4c01-915c-2ed9d421e4e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '191e745d2d9245da8d2df2e4f9a7814a', 'request_id': 'req-a814892c-b61f-4c01-915c-2ed9d421e4e2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:29.571195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a814892c-b61f-4c01-915c-2ed9d421e4e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '191e745d2d9245da8d2df2e4f9a7814a', 'request_id': 'req-a814892c-b61f-4c01-915c-2ed9d421e4e2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:29.577502 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:29.584927 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5ce2e4cf-17eb-456b-80ea-bef557d8f320 None None] MFA Rules not processed for user `ff241cbd3f7c47328ac5ef2ed4c66e18`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:35:29.587357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5ce2e4cf-17eb-456b-80ea-bef557d8f320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:29.588463 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a814892c-b61f-4c01-915c-2ed9d421e4e2 admin admin] The token cache is being invalidate because project 191e745d2d9245da8d2df2e4f9a7814a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:35:29.588755 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a814892c-b61f-4c01-915c-2ed9d421e4e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:29.589440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a814892c-b61f-4c01-915c-2ed9d421e4e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:35:29.591289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1208/2407] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:35:29 2022] DELETE /identity/v3/projects/191e745d2d9245da8d2df2e4f9a7814a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:35:29.661231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1200/2408] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:35:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:35:39.053148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327df0ef-4e48-4d83-8d93-43da0d7de0a8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:39.053674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327df0ef-4e48-4d83-8d93-43da0d7de0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:39.053674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327df0ef-4e48-4d83-8d93-43da0d7de0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:39.073709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-327df0ef-4e48-4d83-8d93-43da0d7de0a8 None None] MFA Rules not processed for user `5bb515269d1a45f594f3d297c7fd267e`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:35:39.074904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-327df0ef-4e48-4d83-8d93-43da0d7de0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:39.132422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1209/2409] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:35:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:35:39.652007 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=22 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:49.125383 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:50.867618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2e4cf-17eb-456b-80ea-bef557d8f320 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:50.869392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5ce2e4cf-17eb-456b-80ea-bef557d8f320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:50.965061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2e4cf-17eb-456b-80ea-bef557d8f320 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:50.971728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00a657-4bad-4dbf-b0e8-244f2fb24f81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:50.972408 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00a657-4bad-4dbf-b0e8-244f2fb24f81 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:50.972773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00a657-4bad-4dbf-b0e8-244f2fb24f81 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:50.973305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00a657-4bad-4dbf-b0e8-244f2fb24f81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:50.975124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-de00a657-4bad-4dbf-b0e8-244f2fb24f81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:50.985887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00a657-4bad-4dbf-b0e8-244f2fb24f81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:50.987724 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00a657-4bad-4dbf-b0e8-244f2fb24f81 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:50.997222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1201/2410] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:35:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:35:51.781110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e03702-0909-4258-8710-a5c906702bb6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:51.781763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e03702-0909-4258-8710-a5c906702bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:51.781892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e03702-0909-4258-8710-a5c906702bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:51.801879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d4e03702-0909-4258-8710-a5c906702bb6 None None] MFA Rules not processed for user `ed4e695113f843fda52ce62792a36bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:35:51.803075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d4e03702-0909-4258-8710-a5c906702bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:51.858499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1210/2411] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:35:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:35:52.173620 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00a657-4bad-4dbf-b0e8-244f2fb24f81 service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:35:52.175208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-de00a657-4bad-4dbf-b0e8-244f2fb24f81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:52.234086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00a657-4bad-4dbf-b0e8-244f2fb24f81 service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:35:52.237816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9510e0-ba5b-419b-aacc-bb647def1e03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:35:52.238424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9510e0-ba5b-419b-aacc-bb647def1e03 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:35:52.238705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9510e0-ba5b-419b-aacc-bb647def1e03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:35:52.238964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9510e0-ba5b-419b-aacc-bb647def1e03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:35:52.240782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7e9510e0-ba5b-419b-aacc-bb647def1e03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:35:52.251379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9510e0-ba5b-419b-aacc-bb647def1e03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:35:52.252798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9510e0-ba5b-419b-aacc-bb647def1e03 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:35:52.260832 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1202/2412] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:35:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:36:01.852184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=25 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:36:02.257619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=60 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:36:59.212414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e03702-0909-4258-8710-a5c906702bb6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:36:59.213948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d4e03702-0909-4258-8710-a5c906702bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:36:59.356797 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e03702-0909-4258-8710-a5c906702bb6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:36:59.362218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb922f-e3c1-45e7-9e78-fec06d8d2f0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:36:59.362929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb922f-e3c1-45e7-9e78-fec06d8d2f0d service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:36:59.363208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb922f-e3c1-45e7-9e78-fec06d8d2f0d service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:36:59.363471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb922f-e3c1-45e7-9e78-fec06d8d2f0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:36:59.365282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-91bb922f-e3c1-45e7-9e78-fec06d8d2f0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:36:59.376585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb922f-e3c1-45e7-9e78-fec06d8d2f0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:36:59.378259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb922f-e3c1-45e7-9e78-fec06d8d2f0d service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:36:59.387987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1211/2413] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:36:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:09.384159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=63 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:11.550919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9510e0-ba5b-419b-aacc-bb647def1e03 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:11.552393 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7e9510e0-ba5b-419b-aacc-bb647def1e03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:11.616157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9510e0-ba5b-419b-aacc-bb647def1e03 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:11.619903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19408daf-9556-4114-88cb-3ab5b9430ef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:11.620518 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19408daf-9556-4114-88cb-3ab5b9430ef6 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:11.620802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19408daf-9556-4114-88cb-3ab5b9430ef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:11.621074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19408daf-9556-4114-88cb-3ab5b9430ef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:11.622891 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-19408daf-9556-4114-88cb-3ab5b9430ef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:11.711566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19408daf-9556-4114-88cb-3ab5b9430ef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:11.713133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19408daf-9556-4114-88cb-3ab5b9430ef6 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:11.733105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1203/2414] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:15.022689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb922f-e3c1-45e7-9e78-fec06d8d2f0d service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:15.024307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-91bb922f-e3c1-45e7-9e78-fec06d8d2f0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:15.038837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb922f-e3c1-45e7-9e78-fec06d8d2f0d service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:15.043808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc764b-3fc1-459e-9ed4-215b25ba2d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:15.044823 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc764b-3fc1-459e-9ed4-215b25ba2d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:15.044823 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc764b-3fc1-459e-9ed4-215b25ba2d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:15.044997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc764b-3fc1-459e-9ed4-215b25ba2d18 admin admin] PATH_INFO: `/v3/users/5bb515269d1a45f594f3d297c7fd267e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:15.046148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc764b-3fc1-459e-9ed4-215b25ba2d18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bb515269d1a45f594f3d297c7fd267e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:15.048504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc764b-3fc1-459e-9ed4-215b25ba2d18 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:15.100521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-aacc764b-3fc1-459e-9ed4-215b25ba2d18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bb515269d1a45f594f3d297c7fd267e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:15.110130 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-aacc764b-3fc1-459e-9ed4-215b25ba2d18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bb515269d1a45f594f3d297c7fd267e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:15.110781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-aacc764b-3fc1-459e-9ed4-215b25ba2d18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bb515269d1a45f594f3d297c7fd267e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:15.120600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-aacc764b-3fc1-459e-9ed4-215b25ba2d18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bb515269d1a45f594f3d297c7fd267e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:15.138860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-aacc764b-3fc1-459e-9ed4-215b25ba2d18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bb515269d1a45f594f3d297c7fd267e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:15.144953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1212/2415] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:37:15 2022] DELETE /identity/v3/users/5bb515269d1a45f594f3d297c7fd267e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:15.333062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19408daf-9556-4114-88cb-3ab5b9430ef6 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:15.343463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19408daf-9556-4114-88cb-3ab5b9430ef6 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:15.401495 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed73897-3777-4bd1-82a9-ad89772a6c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:15.402477 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed73897-3777-4bd1-82a9-ad89772a6c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:15.402818 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed73897-3777-4bd1-82a9-ad89772a6c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:15.403085 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed73897-3777-4bd1-82a9-ad89772a6c99 admin admin] PATH_INFO: `/v3/projects/d54f5f66e0eb4be9aad2e6d37e0b6db6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:15.404315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed73897-3777-4bd1-82a9-ad89772a6c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d54f5f66e0eb4be9aad2e6d37e0b6db6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:15.406349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed73897-3777-4bd1-82a9-ad89772a6c99 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:15.429384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:15.458759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bed73897-3777-4bd1-82a9-ad89772a6c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'request_id': 'req-bed73897-3777-4bd1-82a9-ad89772a6c99'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:15.467398 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bed73897-3777-4bd1-82a9-ad89772a6c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'request_id': 'req-bed73897-3777-4bd1-82a9-ad89772a6c99'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:15.473277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bed73897-3777-4bd1-82a9-ad89772a6c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'request_id': 'req-bed73897-3777-4bd1-82a9-ad89772a6c99'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:15.485120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bed73897-3777-4bd1-82a9-ad89772a6c99 admin admin] The token cache is being invalidate because project d54f5f66e0eb4be9aad2e6d37e0b6db6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:37:15.485428 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bed73897-3777-4bd1-82a9-ad89772a6c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:15.486095 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bed73897-3777-4bd1-82a9-ad89772a6c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:15.487661 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1204/2416] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:37:15 2022] DELETE /identity/v3/projects/d54f5f66e0eb4be9aad2e6d37e0b6db6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:15.492935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece57f3-d2cd-4071-8b6e-309718bcf756 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:15.493279 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece57f3-d2cd-4071-8b6e-309718bcf756 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:15.493540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece57f3-d2cd-4071-8b6e-309718bcf756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:15.551626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5ece57f3-d2cd-4071-8b6e-309718bcf756 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:15.552851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5ece57f3-d2cd-4071-8b6e-309718bcf756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:15.612583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1213/2417] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:15.629060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d1821-2365-417e-b442-73f7cc6c9197 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:15.629488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d1821-2365-417e-b442-73f7cc6c9197 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:15.629829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d1821-2365-417e-b442-73f7cc6c9197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:15.654292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-277d1821-2365-417e-b442-73f7cc6c9197 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:15.655477 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-277d1821-2365-417e-b442-73f7cc6c9197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:15.664474 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1205/2418] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:15.668918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece57f3-d2cd-4071-8b6e-309718bcf756 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:15.679080 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece57f3-d2cd-4071-8b6e-309718bcf756 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:15.684958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05734a5c-3540-4d8c-80d3-937e49d05abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:15.684958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05734a5c-3540-4d8c-80d3-937e49d05abc admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:15.685230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05734a5c-3540-4d8c-80d3-937e49d05abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:15.685437 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05734a5c-3540-4d8c-80d3-937e49d05abc admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:15.686079 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05734a5c-3540-4d8c-80d3-937e49d05abc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:37:15.686378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05734a5c-3540-4d8c-80d3-937e49d05abc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:15.687982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05734a5c-3540-4d8c-80d3-937e49d05abc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:15.700157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1214/2419] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:37:15.704572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d1821-2365-417e-b442-73f7cc6c9197 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:15.713821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d1821-2365-417e-b442-73f7cc6c9197 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:15.719259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e273bc1-4838-4ff7-a7f1-6b057c3922a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:15.719797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e273bc1-4838-4ff7-a7f1-6b057c3922a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:15.720072 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e273bc1-4838-4ff7-a7f1-6b057c3922a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:15.720337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e273bc1-4838-4ff7-a7f1-6b057c3922a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:15.720931 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e273bc1-4838-4ff7-a7f1-6b057c3922a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:15.726756 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e273bc1-4838-4ff7-a7f1-6b057c3922a7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:15.755933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1206/2420] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:37:15 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:15.760868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05734a5c-3540-4d8c-80d3-937e49d05abc admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:15.771752 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05734a5c-3540-4d8c-80d3-937e49d05abc admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:15.798797 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:15.836481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebbfc1-d3c1-473f-be5e-94de9268ab1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:15.837093 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebbfc1-d3c1-473f-be5e-94de9268ab1b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:15.837404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebbfc1-d3c1-473f-be5e-94de9268ab1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:15.837669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebbfc1-d3c1-473f-be5e-94de9268ab1b admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:15.838305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebbfc1-d3c1-473f-be5e-94de9268ab1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:15.840268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebbfc1-d3c1-473f-be5e-94de9268ab1b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:15.863401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1215/2421] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:37:15 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:37:15.867622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e273bc1-4838-4ff7-a7f1-6b057c3922a7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:15.878447 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e273bc1-4838-4ff7-a7f1-6b057c3922a7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:15.884345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847264bc-5dc8-4a55-89f0-8ee26c4364f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:15.884941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847264bc-5dc8-4a55-89f0-8ee26c4364f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:15.885248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847264bc-5dc8-4a55-89f0-8ee26c4364f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:15.885512 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847264bc-5dc8-4a55-89f0-8ee26c4364f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:15.886152 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847264bc-5dc8-4a55-89f0-8ee26c4364f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:15.887859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847264bc-5dc8-4a55-89f0-8ee26c4364f8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:15.898561 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1207/2422] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:15.902786 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebbfc1-d3c1-473f-be5e-94de9268ab1b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:15.913036 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebbfc1-d3c1-473f-be5e-94de9268ab1b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:15.917817 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58b283-0897-497d-ba8f-0cd4f1960897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:15.918472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58b283-0897-497d-ba8f-0cd4f1960897 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:15.918746 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58b283-0897-497d-ba8f-0cd4f1960897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:15.919011 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58b283-0897-497d-ba8f-0cd4f1960897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:15.919704 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58b283-0897-497d-ba8f-0cd4f1960897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:15.921328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58b283-0897-497d-ba8f-0cd4f1960897 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:15.931214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1216/2423] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:37:15.935344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847264bc-5dc8-4a55-89f0-8ee26c4364f8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:15.945098 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847264bc-5dc8-4a55-89f0-8ee26c4364f8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:15.949827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf49e1-6916-4623-a805-4bd40dbedb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:15.950510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf49e1-6916-4623-a805-4bd40dbedb18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:15.950768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf49e1-6916-4623-a805-4bd40dbedb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:15.951034 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf49e1-6916-4623-a805-4bd40dbedb18 admin admin] PATH_INFO: `/v3/projects/928e619eeefb45bf8e8597df083b1fce/users/9af292fc356c4276b70d718089051eb2/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:15.964993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf49e1-6916-4623-a805-4bd40dbedb18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=928e619eeefb45bf8e8597df083b1fce, user_id=9af292fc356c4276b70d718089051eb2, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:15.968030 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf49e1-6916-4623-a805-4bd40dbedb18 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:15.982457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1208/2424] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:15 2022] PUT /identity/v3/projects/928e619eeefb45bf8e8597df083b1fce/users/9af292fc356c4276b70d718089051eb2/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:15.992918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1758c29-8e26-4e47-b8cb-eadf55b79379 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:15.993263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1758c29-8e26-4e47-b8cb-eadf55b79379 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:15.993539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1758c29-8e26-4e47-b8cb-eadf55b79379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:16.021202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a1758c29-8e26-4e47-b8cb-eadf55b79379 None None] MFA Rules not processed for user `9af292fc356c4276b70d718089051eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:16.022383 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a1758c29-8e26-4e47-b8cb-eadf55b79379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:16.092515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1217/2425] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:16.104281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf49e1-6916-4623-a805-4bd40dbedb18 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:16.106250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b6cf49e1-6916-4623-a805-4bd40dbedb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:16.210100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf49e1-6916-4623-a805-4bd40dbedb18 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:16.217606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22a365-ccb9-4f9e-bd13-9a37b24f3f70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:16.218575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22a365-ccb9-4f9e-bd13-9a37b24f3f70 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:16.218860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22a365-ccb9-4f9e-bd13-9a37b24f3f70 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:16.219193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22a365-ccb9-4f9e-bd13-9a37b24f3f70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:16.231230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22a365-ccb9-4f9e-bd13-9a37b24f3f70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:16.233675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22a365-ccb9-4f9e-bd13-9a37b24f3f70 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:16.247080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1209/2426] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:18.962792 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1758c29-8e26-4e47-b8cb-eadf55b79379 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:19.028525 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1758c29-8e26-4e47-b8cb-eadf55b79379 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:19.033630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:19.034277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:19.034555 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:19.034823 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] PATH_INFO: `/v3/users/9af292fc356c4276b70d718089051eb2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:19.036040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af292fc356c4276b70d718089051eb2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:19.038047 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:19.090874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:19.098439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af292fc356c4276b70d718089051eb2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:19.106792 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af292fc356c4276b70d718089051eb2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:19.107478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af292fc356c4276b70d718089051eb2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:19.114396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af292fc356c4276b70d718089051eb2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:19.126440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af292fc356c4276b70d718089051eb2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:19.132584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1218/2427] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:37:18 2022] DELETE /identity/v3/users/9af292fc356c4276b70d718089051eb2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:19.137368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccc939-c63d-440f-8f58-af0c2f92c257 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:19.137847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccc939-c63d-440f-8f58-af0c2f92c257 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:19.137942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccc939-c63d-440f-8f58-af0c2f92c257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:19.144952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:19.159683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d5ccc939-c63d-440f-8f58-af0c2f92c257 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:19.160881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d5ccc939-c63d-440f-8f58-af0c2f92c257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:19.219609 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1210/2428] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:19.231331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:19.233049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:19.293800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4e3e-b6d1-4b7b-a297-1f5e28ec38c9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:19.297676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635d824-7280-4fec-9bd9-6094719385fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:19.298286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635d824-7280-4fec-9bd9-6094719385fe service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:19.298562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635d824-7280-4fec-9bd9-6094719385fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:19.298832 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635d824-7280-4fec-9bd9-6094719385fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:19.308273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635d824-7280-4fec-9bd9-6094719385fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:19.309834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635d824-7280-4fec-9bd9-6094719385fe service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:19.319306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1219/2429] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:19.604071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ccc939-c63d-440f-8f58-af0c2f92c257 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:19.605625 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d5ccc939-c63d-440f-8f58-af0c2f92c257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:19.620516 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ccc939-c63d-440f-8f58-af0c2f92c257 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:19.625985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce9eab-7527-43f0-8519-e9aa643a71f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:19.627056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce9eab-7527-43f0-8519-e9aa643a71f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:19.627360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce9eab-7527-43f0-8519-e9aa643a71f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:19.627661 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce9eab-7527-43f0-8519-e9aa643a71f3 admin admin] PATH_INFO: `/v3/projects/928e619eeefb45bf8e8597df083b1fce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:19.628761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce9eab-7527-43f0-8519-e9aa643a71f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=928e619eeefb45bf8e8597df083b1fce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:19.630775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce9eab-7527-43f0-8519-e9aa643a71f3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:19.682001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-74ce9eab-7527-43f0-8519-e9aa643a71f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '928e619eeefb45bf8e8597df083b1fce', 'request_id': 'req-74ce9eab-7527-43f0-8519-e9aa643a71f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:19.690372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-74ce9eab-7527-43f0-8519-e9aa643a71f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '928e619eeefb45bf8e8597df083b1fce', 'request_id': 'req-74ce9eab-7527-43f0-8519-e9aa643a71f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:19.696388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-74ce9eab-7527-43f0-8519-e9aa643a71f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '928e619eeefb45bf8e8597df083b1fce', 'request_id': 'req-74ce9eab-7527-43f0-8519-e9aa643a71f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:19.708102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-74ce9eab-7527-43f0-8519-e9aa643a71f3 admin admin] The token cache is being invalidate because project 928e619eeefb45bf8e8597df083b1fce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:37:19.708450 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-74ce9eab-7527-43f0-8519-e9aa643a71f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:19.709124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-74ce9eab-7527-43f0-8519-e9aa643a71f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:19.711058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1211/2430] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:37:19 2022] DELETE /identity/v3/projects/928e619eeefb45bf8e8597df083b1fce => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:19.716004 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea03fab-bc79-4116-924b-9e0668cb89c4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:19.716829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea03fab-bc79-4116-924b-9e0668cb89c4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:19.717212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea03fab-bc79-4116-924b-9e0668cb89c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:19.737819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8ea03fab-bc79-4116-924b-9e0668cb89c4 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:19.739107 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8ea03fab-bc79-4116-924b-9e0668cb89c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:19.804734 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1220/2431] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:19.814081 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786cea3-8ea7-4c7f-9d13-d8261a2d7092 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:19.814500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786cea3-8ea7-4c7f-9d13-d8261a2d7092 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:19.814901 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786cea3-8ea7-4c7f-9d13-d8261a2d7092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:19.835559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9786cea3-8ea7-4c7f-9d13-d8261a2d7092 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:19.836792 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9786cea3-8ea7-4c7f-9d13-d8261a2d7092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:19.845399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1212/2432] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:19.849840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea03fab-bc79-4116-924b-9e0668cb89c4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:19.859544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea03fab-bc79-4116-924b-9e0668cb89c4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:19.864096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444728a7-0ee7-40bf-b21a-ab05ed537b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:19.864700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444728a7-0ee7-40bf-b21a-ab05ed537b75 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:19.864977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444728a7-0ee7-40bf-b21a-ab05ed537b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:19.865286 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444728a7-0ee7-40bf-b21a-ab05ed537b75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:19.865934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444728a7-0ee7-40bf-b21a-ab05ed537b75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:37:19.866272 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444728a7-0ee7-40bf-b21a-ab05ed537b75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:19.867861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444728a7-0ee7-40bf-b21a-ab05ed537b75 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:19.880434 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1221/2433] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:37:19.883896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786cea3-8ea7-4c7f-9d13-d8261a2d7092 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:19.894256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786cea3-8ea7-4c7f-9d13-d8261a2d7092 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:19.899229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2a7f7-b37e-4edf-b6da-0f3cc52e6417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:19.899867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2a7f7-b37e-4edf-b6da-0f3cc52e6417 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:19.900152 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2a7f7-b37e-4edf-b6da-0f3cc52e6417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:19.900422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2a7f7-b37e-4edf-b6da-0f3cc52e6417 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:19.901070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2a7f7-b37e-4edf-b6da-0f3cc52e6417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:19.903126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2a7f7-b37e-4edf-b6da-0f3cc52e6417 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:19.936384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1213/2434] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:37:19 2022] POST /identity/v3/projects => generated 401 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:37:19.940618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444728a7-0ee7-40bf-b21a-ab05ed537b75 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:19.950597 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444728a7-0ee7-40bf-b21a-ab05ed537b75 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:19.995696 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:20.011189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3d327-275a-4172-9a84-fd00facb0361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:20.011920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3d327-275a-4172-9a84-fd00facb0361 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.012201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3d327-275a-4172-9a84-fd00facb0361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.012471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3d327-275a-4172-9a84-fd00facb0361 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.013145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3d327-275a-4172-9a84-fd00facb0361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:20.015176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3d327-275a-4172-9a84-fd00facb0361 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:20.037285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1222/2435] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:37:19 2022] POST /identity/v3/users => generated 301 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:20.041558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2a7f7-b37e-4edf-b6da-0f3cc52e6417 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:20.051878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2a7f7-b37e-4edf-b6da-0f3cc52e6417 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:20.057490 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5706a36-23c3-4b86-98c3-c0ba5f8b686a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:20.058111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5706a36-23c3-4b86-98c3-c0ba5f8b686a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.058386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5706a36-23c3-4b86-98c3-c0ba5f8b686a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.058658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5706a36-23c3-4b86-98c3-c0ba5f8b686a admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.059408 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5706a36-23c3-4b86-98c3-c0ba5f8b686a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:20.061005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5706a36-23c3-4b86-98c3-c0ba5f8b686a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:20.071274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1214/2436] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:37:20.075930 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3d327-275a-4172-9a84-fd00facb0361 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:20.089719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3d327-275a-4172-9a84-fd00facb0361 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:20.098953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08565699-fe33-4172-9fe9-5b3d0328c266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:20.099725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08565699-fe33-4172-9fe9-5b3d0328c266 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.100046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08565699-fe33-4172-9fe9-5b3d0328c266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.100312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08565699-fe33-4172-9fe9-5b3d0328c266 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.101319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08565699-fe33-4172-9fe9-5b3d0328c266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:20.103572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08565699-fe33-4172-9fe9-5b3d0328c266 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:20.115639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1223/2437] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:20.119992 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5706a36-23c3-4b86-98c3-c0ba5f8b686a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:20.130502 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5706a36-23c3-4b86-98c3-c0ba5f8b686a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:20.136019 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212aa9d-16a4-4b38-899a-0aeeb3bbc4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:20.136659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212aa9d-16a4-4b38-899a-0aeeb3bbc4f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.136932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212aa9d-16a4-4b38-899a-0aeeb3bbc4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.137311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212aa9d-16a4-4b38-899a-0aeeb3bbc4f9 admin admin] PATH_INFO: `/v3/projects/1d3bf900dd884f3c82f7e8293c44105a/users/936882e9accf41a2a7305daf413f632c/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.152243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212aa9d-16a4-4b38-899a-0aeeb3bbc4f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3bf900dd884f3c82f7e8293c44105a, user_id=936882e9accf41a2a7305daf413f632c, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:20.154914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212aa9d-16a4-4b38-899a-0aeeb3bbc4f9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:20.169177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1215/2438] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:20 2022] PUT /identity/v3/projects/1d3bf900dd884f3c82f7e8293c44105a/users/936882e9accf41a2a7305daf413f632c/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:20.179891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330f9ba-a141-4d0a-b69e-18479a76bd77 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.180289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330f9ba-a141-4d0a-b69e-18479a76bd77 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.180570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330f9ba-a141-4d0a-b69e-18479a76bd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.210148 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1330f9ba-a141-4d0a-b69e-18479a76bd77 None None] MFA Rules not processed for user `936882e9accf41a2a7305daf413f632c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:20.211457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1330f9ba-a141-4d0a-b69e-18479a76bd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:20.290233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1224/2439] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:20 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:20.295062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212aa9d-16a4-4b38-899a-0aeeb3bbc4f9 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:20.321629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:20.361462 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212aa9d-16a4-4b38-899a-0aeeb3bbc4f9 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:20.366586 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e28f1-f7ff-452b-946d-f0c1dc0baec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:20.367193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e28f1-f7ff-452b-946d-f0c1dc0baec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.367798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e28f1-f7ff-452b-946d-f0c1dc0baec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.368139 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e28f1-f7ff-452b-946d-f0c1dc0baec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.368745 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e28f1-f7ff-452b-946d-f0c1dc0baec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:20.370677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e28f1-f7ff-452b-946d-f0c1dc0baec8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:20.402453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1216/2440] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:37:20 2022] POST /identity/v3/projects => generated 401 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:20.403772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330f9ba-a141-4d0a-b69e-18479a76bd77 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:20.414858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330f9ba-a141-4d0a-b69e-18479a76bd77 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:20.476773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21494e-dd9a-47a1-a426-67f52c212ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:20.477466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21494e-dd9a-47a1-a426-67f52c212ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.477740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21494e-dd9a-47a1-a426-67f52c212ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.478002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21494e-dd9a-47a1-a426-67f52c212ab8 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.478628 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a21494e-dd9a-47a1-a426-67f52c212ab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:20.480533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a21494e-dd9a-47a1-a426-67f52c212ab8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:20.507890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1225/2441] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:37:20 2022] POST /identity/v3/users => generated 307 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:37:20.512286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e28f1-f7ff-452b-946d-f0c1dc0baec8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:20.528131 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e28f1-f7ff-452b-946d-f0c1dc0baec8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:20.533697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2ad7b-7034-41af-9a70-4d92a5c4d615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:20.534327 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2ad7b-7034-41af-9a70-4d92a5c4d615 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.534599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2ad7b-7034-41af-9a70-4d92a5c4d615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.534863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2ad7b-7034-41af-9a70-4d92a5c4d615 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.535503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2ad7b-7034-41af-9a70-4d92a5c4d615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:20.537113 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2ad7b-7034-41af-9a70-4d92a5c4d615 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:20.548305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1217/2442] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:20.553413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21494e-dd9a-47a1-a426-67f52c212ab8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:20.564193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21494e-dd9a-47a1-a426-67f52c212ab8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:20.569852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7425a-6f6a-4307-a011-30ce600e39e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:20.570511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7425a-6f6a-4307-a011-30ce600e39e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.570798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7425a-6f6a-4307-a011-30ce600e39e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.571067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7425a-6f6a-4307-a011-30ce600e39e6 admin admin] PATH_INFO: `/v3/projects/adca5389b46141db843390550eb3b8bb/users/c8488530f5c049b4845918b603a8f39e/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.586974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7425a-6f6a-4307-a011-30ce600e39e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adca5389b46141db843390550eb3b8bb, user_id=c8488530f5c049b4845918b603a8f39e, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:20.589678 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7425a-6f6a-4307-a011-30ce600e39e6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:20.606179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1226/2443] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:20 2022] PUT /identity/v3/projects/adca5389b46141db843390550eb3b8bb/users/c8488530f5c049b4845918b603a8f39e/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:20.610292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2ad7b-7034-41af-9a70-4d92a5c4d615 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:20.625375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2ad7b-7034-41af-9a70-4d92a5c4d615 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:20.690487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3de3e-93de-4294-876c-b14ec0f647f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:20.691255 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3de3e-93de-4294-876c-b14ec0f647f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.691542 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3de3e-93de-4294-876c-b14ec0f647f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.691815 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3de3e-93de-4294-876c-b14ec0f647f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.692456 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3de3e-93de-4294-876c-b14ec0f647f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:20.694100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3de3e-93de-4294-876c-b14ec0f647f2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:20.704858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1218/2444] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:20.709288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7425a-6f6a-4307-a011-30ce600e39e6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:20.720305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7425a-6f6a-4307-a011-30ce600e39e6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:20.725904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20165268-794f-43a3-aa2b-dc3137e27b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:20.726588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20165268-794f-43a3-aa2b-dc3137e27b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.726855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20165268-794f-43a3-aa2b-dc3137e27b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.727161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20165268-794f-43a3-aa2b-dc3137e27b9b admin admin] PATH_INFO: `/v3/projects/adca5389b46141db843390550eb3b8bb/users/c8488530f5c049b4845918b603a8f39e/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.729363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20165268-794f-43a3-aa2b-dc3137e27b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=adca5389b46141db843390550eb3b8bb, user_id=c8488530f5c049b4845918b603a8f39e, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:20.732107 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20165268-794f-43a3-aa2b-dc3137e27b9b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:20.743316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1227/2445] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:20 2022] PUT /identity/v3/projects/adca5389b46141db843390550eb3b8bb/users/c8488530f5c049b4845918b603a8f39e/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:37:20.753814 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b129a0-84fc-4e6b-bbae-416ee5bbca36 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.754097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b129a0-84fc-4e6b-bbae-416ee5bbca36 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.754392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b129a0-84fc-4e6b-bbae-416ee5bbca36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.785457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-10b129a0-84fc-4e6b-bbae-416ee5bbca36 None None] MFA Rules not processed for user `c8488530f5c049b4845918b603a8f39e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:20.786679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-10b129a0-84fc-4e6b-bbae-416ee5bbca36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:20.861269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1219/2446] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:20 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:20.874111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20165268-794f-43a3-aa2b-dc3137e27b9b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:20.876181 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-20165268-794f-43a3-aa2b-dc3137e27b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:20.888728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:20.962668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20165268-794f-43a3-aa2b-dc3137e27b9b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:20.966882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af207e-bd54-4038-bc05-56d7261e6360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:20.967486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af207e-bd54-4038-bc05-56d7261e6360 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:20.967761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af207e-bd54-4038-bc05-56d7261e6360 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:20.968027 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af207e-bd54-4038-bc05-56d7261e6360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:20.977011 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af207e-bd54-4038-bc05-56d7261e6360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:20.978471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af207e-bd54-4038-bc05-56d7261e6360 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:20.988274 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1228/2447] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:20 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:21.070557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b129a0-84fc-4e6b-bbae-416ee5bbca36 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:21.080488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b129a0-84fc-4e6b-bbae-416ee5bbca36 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:21.085041 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e280b-742e-4892-857e-696a4c3c622c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:21.085675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e280b-742e-4892-857e-696a4c3c622c service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:21.085952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e280b-742e-4892-857e-696a4c3c622c service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:21.086218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e280b-742e-4892-857e-696a4c3c622c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:21.101384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:21.149372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e280b-742e-4892-857e-696a4c3c622c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:21.150821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e280b-742e-4892-857e-696a4c3c622c service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:21.159762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1220/2448] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:21 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:25.301377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af207e-bd54-4038-bc05-56d7261e6360 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:25.374760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af207e-bd54-4038-bc05-56d7261e6360 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:25.379277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:25.379908 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:25.380177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:25.380448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] PATH_INFO: `/v3/users/936882e9accf41a2a7305daf413f632c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:25.381627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=936882e9accf41a2a7305daf413f632c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:25.383638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:25.431883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '936882e9accf41a2a7305daf413f632c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:25.438843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '936882e9accf41a2a7305daf413f632c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:25.439513 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '936882e9accf41a2a7305daf413f632c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:25.446516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '936882e9accf41a2a7305daf413f632c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:25.474307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '936882e9accf41a2a7305daf413f632c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:25.482115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1229/2449] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:37:25 2022] DELETE /identity/v3/users/936882e9accf41a2a7305daf413f632c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:25.486560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28510b6e-b75e-4f27-b525-1bf407a54061 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:25.487002 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28510b6e-b75e-4f27-b525-1bf407a54061 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:25.487101 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28510b6e-b75e-4f27-b525-1bf407a54061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:25.506454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-28510b6e-b75e-4f27-b525-1bf407a54061 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:25.507656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-28510b6e-b75e-4f27-b525-1bf407a54061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:25.574275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1221/2450] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:25.580997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:25.582563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:25.593728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:25.642307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f9efd-9c6a-4bb3-9ca1-f96a3340bc12 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:25.646262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833adda-c8ee-4dd1-a394-1af587db6541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:25.646871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833adda-c8ee-4dd1-a394-1af587db6541 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:25.647141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833adda-c8ee-4dd1-a394-1af587db6541 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:25.647409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833adda-c8ee-4dd1-a394-1af587db6541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:25.656652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833adda-c8ee-4dd1-a394-1af587db6541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:25.658087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833adda-c8ee-4dd1-a394-1af587db6541 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:25.668404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1230/2451] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:25.962585 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28510b6e-b75e-4f27-b525-1bf407a54061 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:25.964275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-28510b6e-b75e-4f27-b525-1bf407a54061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:25.978111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28510b6e-b75e-4f27-b525-1bf407a54061 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:25.983987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:25.984643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:25.984918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:25.985228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] PATH_INFO: `/v3/projects/1d3bf900dd884f3c82f7e8293c44105a` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:25.986484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d3bf900dd884f3c82f7e8293c44105a)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:25.988418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:26.056724 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d3bf900dd884f3c82f7e8293c44105a', 'request_id': 'req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.065967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d3bf900dd884f3c82f7e8293c44105a', 'request_id': 'req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.072305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d3bf900dd884f3c82f7e8293c44105a', 'request_id': 'req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.083498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] The token cache is being invalidate because project 1d3bf900dd884f3c82f7e8293c44105a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:37:26.083787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.084435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.086063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1222/2452] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:37:25 2022] DELETE /identity/v3/projects/1d3bf900dd884f3c82f7e8293c44105a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:26.090005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833adda-c8ee-4dd1-a394-1af587db6541 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:26.099776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833adda-c8ee-4dd1-a394-1af587db6541 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:26.155398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f73d24-2d89-4d20-9a66-3f8b24616fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:26.156036 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f73d24-2d89-4d20-9a66-3f8b24616fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:26.156310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f73d24-2d89-4d20-9a66-3f8b24616fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:26.156583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f73d24-2d89-4d20-9a66-3f8b24616fca admin admin] PATH_INFO: `/v3/users/c8488530f5c049b4845918b603a8f39e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:26.157742 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f73d24-2d89-4d20-9a66-3f8b24616fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8488530f5c049b4845918b603a8f39e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:26.159675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f73d24-2d89-4d20-9a66-3f8b24616fca admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:26.209303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c2f73d24-2d89-4d20-9a66-3f8b24616fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8488530f5c049b4845918b603a8f39e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.218401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c2f73d24-2d89-4d20-9a66-3f8b24616fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8488530f5c049b4845918b603a8f39e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.219089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c2f73d24-2d89-4d20-9a66-3f8b24616fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8488530f5c049b4845918b603a8f39e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.226077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c2f73d24-2d89-4d20-9a66-3f8b24616fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8488530f5c049b4845918b603a8f39e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.242828 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c2f73d24-2d89-4d20-9a66-3f8b24616fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8488530f5c049b4845918b603a8f39e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.249147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1231/2453] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:37:26 2022] DELETE /identity/v3/users/c8488530f5c049b4845918b603a8f39e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:26.683593 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:26.685149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:26.698773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56e517-a9ad-4ea3-9b3a-c670e39368dd admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:26.750223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:26.773311 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d5b40-24b8-4d6e-82f2-efe04426f906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:26.774052 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d5b40-24b8-4d6e-82f2-efe04426f906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:26.774395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d5b40-24b8-4d6e-82f2-efe04426f906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:26.774733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d5b40-24b8-4d6e-82f2-efe04426f906 admin admin] PATH_INFO: `/v3/projects/adca5389b46141db843390550eb3b8bb` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:26.775944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d5b40-24b8-4d6e-82f2-efe04426f906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adca5389b46141db843390550eb3b8bb)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:26.777960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d5b40-24b8-4d6e-82f2-efe04426f906 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:26.829898 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d52d5b40-24b8-4d6e-82f2-efe04426f906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adca5389b46141db843390550eb3b8bb', 'request_id': 'req-d52d5b40-24b8-4d6e-82f2-efe04426f906'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.843554 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d52d5b40-24b8-4d6e-82f2-efe04426f906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adca5389b46141db843390550eb3b8bb', 'request_id': 'req-d52d5b40-24b8-4d6e-82f2-efe04426f906'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.850427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d52d5b40-24b8-4d6e-82f2-efe04426f906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adca5389b46141db843390550eb3b8bb', 'request_id': 'req-d52d5b40-24b8-4d6e-82f2-efe04426f906'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.862560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d52d5b40-24b8-4d6e-82f2-efe04426f906 admin admin] The token cache is being invalidate because project adca5389b46141db843390550eb3b8bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:37:26.863091 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d52d5b40-24b8-4d6e-82f2-efe04426f906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.864130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d52d5b40-24b8-4d6e-82f2-efe04426f906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:26.865863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1223/2454] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:37:26 2022] DELETE /identity/v3/projects/adca5389b46141db843390550eb3b8bb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:26.871018 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a6d89-53ea-4f68-bca5-3964fe4e5274 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:26.871517 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a6d89-53ea-4f68-bca5-3964fe4e5274 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:26.871730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a6d89-53ea-4f68-bca5-3964fe4e5274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:26.891823 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-699a6d89-53ea-4f68-bca5-3964fe4e5274 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:26.893034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-699a6d89-53ea-4f68-bca5-3964fe4e5274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:26.920427 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:26.958647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1232/2455] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:26.966706 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0de32a-6303-460e-8f3c-4917c034e58b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:26.966982 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0de32a-6303-460e-8f3c-4917c034e58b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:26.967951 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0de32a-6303-460e-8f3c-4917c034e58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:26.990749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5a0de32a-6303-460e-8f3c-4917c034e58b None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:26.992569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5a0de32a-6303-460e-8f3c-4917c034e58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:27.003117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1224/2456] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:27.007805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a6d89-53ea-4f68-bca5-3964fe4e5274 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:27.018011 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a6d89-53ea-4f68-bca5-3964fe4e5274 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:27.023423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9b27d-5ba0-4b60-bb05-b2f2ef56e27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:27.024043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9b27d-5ba0-4b60-bb05-b2f2ef56e27c admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.024322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9b27d-5ba0-4b60-bb05-b2f2ef56e27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.024590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9b27d-5ba0-4b60-bb05-b2f2ef56e27c admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.025280 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9b27d-5ba0-4b60-bb05-b2f2ef56e27c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:37:27.025582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9b27d-5ba0-4b60-bb05-b2f2ef56e27c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:27.027178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9b27d-5ba0-4b60-bb05-b2f2ef56e27c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:27.039659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1233/2457] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:37:27.044240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0de32a-6303-460e-8f3c-4917c034e58b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:27.054572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0de32a-6303-460e-8f3c-4917c034e58b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:27.060541 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724b3ad-0c75-436c-886e-e4fa659aa376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:27.061193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724b3ad-0c75-436c-886e-e4fa659aa376 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.061596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724b3ad-0c75-436c-886e-e4fa659aa376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.061875 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724b3ad-0c75-436c-886e-e4fa659aa376 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.062517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724b3ad-0c75-436c-886e-e4fa659aa376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:27.064419 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724b3ad-0c75-436c-886e-e4fa659aa376 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:27.097149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1225/2458] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:37:27 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:27.099087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9b27d-5ba0-4b60-bb05-b2f2ef56e27c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:27.117414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9b27d-5ba0-4b60-bb05-b2f2ef56e27c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:27.179932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d02dd3-1497-4a5f-9bbc-9c881b3e4ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:27.180883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d02dd3-1497-4a5f-9bbc-9c881b3e4ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.181359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d02dd3-1497-4a5f-9bbc-9c881b3e4ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.182162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d02dd3-1497-4a5f-9bbc-9c881b3e4ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.182299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d02dd3-1497-4a5f-9bbc-9c881b3e4ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:27.184175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d02dd3-1497-4a5f-9bbc-9c881b3e4ec7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:27.209844 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1234/2459] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:37:27 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:37:27.214149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724b3ad-0c75-436c-886e-e4fa659aa376 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:27.225105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724b3ad-0c75-436c-886e-e4fa659aa376 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:27.230094 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa4983-5cba-44c6-88b9-1904f64705d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:27.230753 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa4983-5cba-44c6-88b9-1904f64705d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.231058 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa4983-5cba-44c6-88b9-1904f64705d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.231303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa4983-5cba-44c6-88b9-1904f64705d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.232146 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa4983-5cba-44c6-88b9-1904f64705d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:27.233827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa4983-5cba-44c6-88b9-1904f64705d4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:27.245876 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1226/2460] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:27.250294 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d02dd3-1497-4a5f-9bbc-9c881b3e4ec7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:27.261037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d02dd3-1497-4a5f-9bbc-9c881b3e4ec7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:27.266553 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa5f8b-f07b-4745-9edb-d68fcdc90a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:27.267474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa5f8b-f07b-4745-9edb-d68fcdc90a0a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.267599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa5f8b-f07b-4745-9edb-d68fcdc90a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.267871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa5f8b-f07b-4745-9edb-d68fcdc90a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.268593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa5f8b-f07b-4745-9edb-d68fcdc90a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:27.270281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa5f8b-f07b-4745-9edb-d68fcdc90a0a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:27.282369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1235/2461] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:37:27.286723 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa4983-5cba-44c6-88b9-1904f64705d4 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:27.298149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa4983-5cba-44c6-88b9-1904f64705d4 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:27.303857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad104f6-f248-4693-b1b2-a3860f8d3ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:27.304652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad104f6-f248-4693-b1b2-a3860f8d3ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.304932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad104f6-f248-4693-b1b2-a3860f8d3ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.305260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad104f6-f248-4693-b1b2-a3860f8d3ed3 admin admin] PATH_INFO: `/v3/projects/7bfa92daf8c74f44bb62d47249e72169/users/9195776991fd4bc5acb83ce7958b226b/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.321631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad104f6-f248-4693-b1b2-a3860f8d3ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bfa92daf8c74f44bb62d47249e72169, user_id=9195776991fd4bc5acb83ce7958b226b, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:27.324600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad104f6-f248-4693-b1b2-a3860f8d3ed3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:27.339438 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1227/2462] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:27 2022] PUT /identity/v3/projects/7bfa92daf8c74f44bb62d47249e72169/users/9195776991fd4bc5acb83ce7958b226b/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:27.350224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5cbd6-c411-49ee-b512-caf921dbe95f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.350707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5cbd6-c411-49ee-b512-caf921dbe95f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.350829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5cbd6-c411-49ee-b512-caf921dbe95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.382952 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-62a5cbd6-c411-49ee-b512-caf921dbe95f None None] MFA Rules not processed for user `9195776991fd4bc5acb83ce7958b226b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:27.384313 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-62a5cbd6-c411-49ee-b512-caf921dbe95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:27.468035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1236/2463] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:27.472884 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad104f6-f248-4693-b1b2-a3860f8d3ed3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:27.536344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad104f6-f248-4693-b1b2-a3860f8d3ed3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:27.541416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dba936-03b3-4685-9cba-17a09cc2dbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:27.542037 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dba936-03b3-4685-9cba-17a09cc2dbbc admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.542316 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dba936-03b3-4685-9cba-17a09cc2dbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.542611 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dba936-03b3-4685-9cba-17a09cc2dbbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.543240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dba936-03b3-4685-9cba-17a09cc2dbbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:27.545345 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dba936-03b3-4685-9cba-17a09cc2dbbc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:27.557251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:27.577567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1228/2464] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:37:27 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:37:27.581554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5cbd6-c411-49ee-b512-caf921dbe95f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:27.593371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5cbd6-c411-49ee-b512-caf921dbe95f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:27.626535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:27.678188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83799847-f698-4ba6-82a7-d2432e96145d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:27.678968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83799847-f698-4ba6-82a7-d2432e96145d admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.679343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83799847-f698-4ba6-82a7-d2432e96145d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.679731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83799847-f698-4ba6-82a7-d2432e96145d admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.680474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83799847-f698-4ba6-82a7-d2432e96145d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:27.682530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83799847-f698-4ba6-82a7-d2432e96145d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:27.712951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1237/2465] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:37:27 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:37:27.717565 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dba936-03b3-4685-9cba-17a09cc2dbbc admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:27.730516 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dba936-03b3-4685-9cba-17a09cc2dbbc admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:27.735297 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ed37e-a44e-448d-8752-236972127a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:27.736097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ed37e-a44e-448d-8752-236972127a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.736370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ed37e-a44e-448d-8752-236972127a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.736665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ed37e-a44e-448d-8752-236972127a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.737544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ed37e-a44e-448d-8752-236972127a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:27.739120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ed37e-a44e-448d-8752-236972127a41 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:27.750252 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1229/2466] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:37:27.754492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83799847-f698-4ba6-82a7-d2432e96145d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:27.764622 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83799847-f698-4ba6-82a7-d2432e96145d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:27.769372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ef162-2013-4d48-bff6-51d80b4deb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:27.770153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ef162-2013-4d48-bff6-51d80b4deb7f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.770423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ef162-2013-4d48-bff6-51d80b4deb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.770685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ef162-2013-4d48-bff6-51d80b4deb7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.771324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ef162-2013-4d48-bff6-51d80b4deb7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:27.772924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ef162-2013-4d48-bff6-51d80b4deb7f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:27.786646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1238/2467] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:27.787543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ed37e-a44e-448d-8752-236972127a41 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:27.797754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ed37e-a44e-448d-8752-236972127a41 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:27.802540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60304a8-e52e-44e3-907d-2c4d7dabc9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:27.803302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60304a8-e52e-44e3-907d-2c4d7dabc9ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.803579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60304a8-e52e-44e3-907d-2c4d7dabc9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.803848 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60304a8-e52e-44e3-907d-2c4d7dabc9ff admin admin] PATH_INFO: `/v3/projects/d8b10ff069994911bec6662364b66998/users/faabddbe1ccb4aeeb45f211a7e52c6a0/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.818361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60304a8-e52e-44e3-907d-2c4d7dabc9ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b10ff069994911bec6662364b66998, user_id=faabddbe1ccb4aeeb45f211a7e52c6a0, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:27.821002 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60304a8-e52e-44e3-907d-2c4d7dabc9ff admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:27.834963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1230/2468] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:27 2022] PUT /identity/v3/projects/d8b10ff069994911bec6662364b66998/users/faabddbe1ccb4aeeb45f211a7e52c6a0/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:27.846817 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4415d2-fad9-4dfc-b3ca-e83541814645 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:27.847207 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4415d2-fad9-4dfc-b3ca-e83541814645 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:27.847548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4415d2-fad9-4dfc-b3ca-e83541814645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:27.875698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ea4415d2-fad9-4dfc-b3ca-e83541814645 None None] MFA Rules not processed for user `faabddbe1ccb4aeeb45f211a7e52c6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:27.876921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ea4415d2-fad9-4dfc-b3ca-e83541814645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:27.942424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1239/2469] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:27.946985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60304a8-e52e-44e3-907d-2c4d7dabc9ff admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:28.010260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60304a8-e52e-44e3-907d-2c4d7dabc9ff admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:28.016781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b408a6b-ac80-4093-bc3f-900ab36c1d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:28.017532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b408a6b-ac80-4093-bc3f-900ab36c1d35 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:28.017819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b408a6b-ac80-4093-bc3f-900ab36c1d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:28.018101 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b408a6b-ac80-4093-bc3f-900ab36c1d35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:28.018735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b408a6b-ac80-4093-bc3f-900ab36c1d35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:28.020637 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b408a6b-ac80-4093-bc3f-900ab36c1d35 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:28.035641 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6fefa-72d6-4340-90f2-4a7f9cc8e6a5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:28.035992 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6fefa-72d6-4340-90f2-4a7f9cc8e6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:28.036328 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6fefa-72d6-4340-90f2-4a7f9cc8e6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:28.051488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1231/2470] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:37:27 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:28.055876 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b408a6b-ac80-4093-bc3f-900ab36c1d35 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:28.061692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-21f6fefa-72d6-4340-90f2-4a7f9cc8e6a5 None None] MFA Rules not processed for user `094abcdcd1f54a658564357056e7849a`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:28.063624 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-21f6fefa-72d6-4340-90f2-4a7f9cc8e6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:28.125707 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1240/2471] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:28.129282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b408a6b-ac80-4093-bc3f-900ab36c1d35 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:28.134619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42c1de-8023-42a2-a3eb-6dbe2f124daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:28.135228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42c1de-8023-42a2-a3eb-6dbe2f124daa admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:28.135514 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42c1de-8023-42a2-a3eb-6dbe2f124daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:28.135782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42c1de-8023-42a2-a3eb-6dbe2f124daa admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:28.136413 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42c1de-8023-42a2-a3eb-6dbe2f124daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:28.138380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42c1de-8023-42a2-a3eb-6dbe2f124daa admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:28.161334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1232/2472] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:37:28 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:37:28.165394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6fefa-72d6-4340-90f2-4a7f9cc8e6a5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:28.176685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6fefa-72d6-4340-90f2-4a7f9cc8e6a5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:28.189229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257987fb-0a45-46af-9309-1cfa40d3e4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:28.189847 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257987fb-0a45-46af-9309-1cfa40d3e4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:28.190175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257987fb-0a45-46af-9309-1cfa40d3e4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:28.190453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257987fb-0a45-46af-9309-1cfa40d3e4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:28.191100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257987fb-0a45-46af-9309-1cfa40d3e4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:28.192701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257987fb-0a45-46af-9309-1cfa40d3e4a6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:28.203228 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1241/2473] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:28.207810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42c1de-8023-42a2-a3eb-6dbe2f124daa admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:28.219408 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42c1de-8023-42a2-a3eb-6dbe2f124daa admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:28.224271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d75f0-e048-4608-bc98-3e6829cc289c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:28.225004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d75f0-e048-4608-bc98-3e6829cc289c admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:28.225378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d75f0-e048-4608-bc98-3e6829cc289c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:28.225739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d75f0-e048-4608-bc98-3e6829cc289c admin admin] PATH_INFO: `/v3/projects/34f72fb800ed41a583b0d414be302e56/users/b8078fd6bea34d2bae61ab07214076e3/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:28.241903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d75f0-e048-4608-bc98-3e6829cc289c admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f72fb800ed41a583b0d414be302e56, user_id=b8078fd6bea34d2bae61ab07214076e3, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:28.244991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d75f0-e048-4608-bc98-3e6829cc289c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:28.260786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1233/2474] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:28 2022] PUT /identity/v3/projects/34f72fb800ed41a583b0d414be302e56/users/b8078fd6bea34d2bae61ab07214076e3/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:28.265039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257987fb-0a45-46af-9309-1cfa40d3e4a6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:28.274848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257987fb-0a45-46af-9309-1cfa40d3e4a6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:28.300038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:28.332445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ba1fb-ca98-42ef-af2f-65b412975454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:28.332991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ba1fb-ca98-42ef-af2f-65b412975454 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:28.333281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ba1fb-ca98-42ef-af2f-65b412975454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:28.333553 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ba1fb-ca98-42ef-af2f-65b412975454 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:28.334263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ba1fb-ca98-42ef-af2f-65b412975454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:28.335870 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ba1fb-ca98-42ef-af2f-65b412975454 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:28.346202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1242/2475] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:28.350440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d75f0-e048-4608-bc98-3e6829cc289c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:28.360842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d75f0-e048-4608-bc98-3e6829cc289c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:28.365537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cca51-2fb6-481c-86b3-9a11d977d2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:28.366208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cca51-2fb6-481c-86b3-9a11d977d2af admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:28.366482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cca51-2fb6-481c-86b3-9a11d977d2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:28.366759 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cca51-2fb6-481c-86b3-9a11d977d2af admin admin] PATH_INFO: `/v3/projects/34f72fb800ed41a583b0d414be302e56/users/b8078fd6bea34d2bae61ab07214076e3/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:28.368652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cca51-2fb6-481c-86b3-9a11d977d2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f72fb800ed41a583b0d414be302e56, user_id=b8078fd6bea34d2bae61ab07214076e3, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:28.371279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cca51-2fb6-481c-86b3-9a11d977d2af admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:28.373592 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:28.383853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1234/2476] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:28 2022] PUT /identity/v3/projects/34f72fb800ed41a583b0d414be302e56/users/b8078fd6bea34d2bae61ab07214076e3/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:28.394248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca64ae0-1447-4b6c-a121-f40d215b13e9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:28.394532 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca64ae0-1447-4b6c-a121-f40d215b13e9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:28.394810 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca64ae0-1447-4b6c-a121-f40d215b13e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:28.424203 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3ca64ae0-1447-4b6c-a121-f40d215b13e9 None None] MFA Rules not processed for user `b8078fd6bea34d2bae61ab07214076e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:28.426076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3ca64ae0-1447-4b6c-a121-f40d215b13e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:28.496170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1243/2477] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:28.507621 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cca51-2fb6-481c-86b3-9a11d977d2af admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:28.509436 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ac3cca51-2fb6-481c-86b3-9a11d977d2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:28.567895 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cca51-2fb6-481c-86b3-9a11d977d2af admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:28.571669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed6e75-f31b-4bd8-b9c2-6316cc29cb1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:28.572300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed6e75-f31b-4bd8-b9c2-6316cc29cb1d service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:28.572573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed6e75-f31b-4bd8-b9c2-6316cc29cb1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:28.572846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed6e75-f31b-4bd8-b9c2-6316cc29cb1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:28.582140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed6e75-f31b-4bd8-b9c2-6316cc29cb1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:28.583635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed6e75-f31b-4bd8-b9c2-6316cc29cb1d service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:28.594870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1235/2478] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:28.645246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca64ae0-1447-4b6c-a121-f40d215b13e9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:28.646789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3ca64ae0-1447-4b6c-a121-f40d215b13e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:28.659087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca64ae0-1447-4b6c-a121-f40d215b13e9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:28.662686 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de333c-4e40-4c5f-a333-ad8c8e4f6c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:28.663297 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de333c-4e40-4c5f-a333-ad8c8e4f6c14 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:28.663576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de333c-4e40-4c5f-a333-ad8c8e4f6c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:28.663843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de333c-4e40-4c5f-a333-ad8c8e4f6c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:28.724689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de333c-4e40-4c5f-a333-ad8c8e4f6c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:28.726628 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de333c-4e40-4c5f-a333-ad8c8e4f6c14 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:28.736302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1244/2479] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:31.745145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed6e75-f31b-4bd8-b9c2-6316cc29cb1d service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:31.807508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed6e75-f31b-4bd8-b9c2-6316cc29cb1d service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:31.812154 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:31.812791 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:31.813070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:31.813424 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] PATH_INFO: `/v3/users/9195776991fd4bc5acb83ce7958b226b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:31.814566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9195776991fd4bc5acb83ce7958b226b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:31.816473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:31.868080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9195776991fd4bc5acb83ce7958b226b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:31.877552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9195776991fd4bc5acb83ce7958b226b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:31.878249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9195776991fd4bc5acb83ce7958b226b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:31.892511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9195776991fd4bc5acb83ce7958b226b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:31.908529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9195776991fd4bc5acb83ce7958b226b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:31.915566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1236/2480] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:37:31 2022] DELETE /identity/v3/users/9195776991fd4bc5acb83ce7958b226b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:31.920209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e396fa-eef7-4fc9-84ab-2ef675c2558f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:31.920728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e396fa-eef7-4fc9-84ab-2ef675c2558f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:31.920728 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e396fa-eef7-4fc9-84ab-2ef675c2558f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:31.941040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e4e396fa-eef7-4fc9-84ab-2ef675c2558f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:31.942257 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e4e396fa-eef7-4fc9-84ab-2ef675c2558f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:31.986415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:32.001692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1245/2481] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:32.011564 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:32.013166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:32.032319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:32.072546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a5989-454f-470a-93df-996500fa072d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:32.076504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ac0d0-73ed-4741-b5ad-5ad217f8d23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:32.077116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ac0d0-73ed-4741-b5ad-5ad217f8d23d service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:32.077848 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ac0d0-73ed-4741-b5ad-5ad217f8d23d service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:32.077848 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ac0d0-73ed-4741-b5ad-5ad217f8d23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:32.086952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ac0d0-73ed-4741-b5ad-5ad217f8d23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:32.088496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ac0d0-73ed-4741-b5ad-5ad217f8d23d service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:32.098650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1237/2482] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:32.518493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e396fa-eef7-4fc9-84ab-2ef675c2558f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:32.519982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e4e396fa-eef7-4fc9-84ab-2ef675c2558f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:32.532722 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e396fa-eef7-4fc9-84ab-2ef675c2558f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:32.538360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:32.538996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:32.539422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:32.539695 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] PATH_INFO: `/v3/projects/7bfa92daf8c74f44bb62d47249e72169` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:32.540948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bfa92daf8c74f44bb62d47249e72169)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:32.542899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:32.589045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bfa92daf8c74f44bb62d47249e72169', 'request_id': 'req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:32.599392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bfa92daf8c74f44bb62d47249e72169', 'request_id': 'req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:32.606282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bfa92daf8c74f44bb62d47249e72169', 'request_id': 'req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:32.617236 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] The token cache is being invalidate because project 7bfa92daf8c74f44bb62d47249e72169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:37:32.617523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:32.618260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:32.620213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1246/2483] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:37:32 2022] DELETE /identity/v3/projects/7bfa92daf8c74f44bb62d47249e72169 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:32.624431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ac0d0-73ed-4741-b5ad-5ad217f8d23d service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:32.634315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ac0d0-73ed-4741-b5ad-5ad217f8d23d service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:32.691467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:32.692103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:32.692382 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:32.692653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] PATH_INFO: `/v3/users/faabddbe1ccb4aeeb45f211a7e52c6a0` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:32.693787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] RBAC: Authorizing `identity:delete_user(user_id=faabddbe1ccb4aeeb45f211a7e52c6a0)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:32.695718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:32.745450 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faabddbe1ccb4aeeb45f211a7e52c6a0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:32.754547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faabddbe1ccb4aeeb45f211a7e52c6a0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:32.755254 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faabddbe1ccb4aeeb45f211a7e52c6a0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:32.762897 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faabddbe1ccb4aeeb45f211a7e52c6a0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:32.775910 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faabddbe1ccb4aeeb45f211a7e52c6a0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:32.783261 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1238/2484] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:37:32 2022] DELETE /identity/v3/users/faabddbe1ccb4aeeb45f211a7e52c6a0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:37:33.066014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:33.067620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:33.082529 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b96d78-ff3c-4ace-af6a-893dd7ff1c5d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:33.140942 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:33.141544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:33.141820 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:33.142092 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] PATH_INFO: `/v3/projects/d8b10ff069994911bec6662364b66998` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:33.143263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b10ff069994911bec6662364b66998)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:33.145252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:33.192372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b10ff069994911bec6662364b66998', 'request_id': 'req-288a5e18-33a3-48c9-bcee-b2c371cbb59d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.201415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b10ff069994911bec6662364b66998', 'request_id': 'req-288a5e18-33a3-48c9-bcee-b2c371cbb59d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.206918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b10ff069994911bec6662364b66998', 'request_id': 'req-288a5e18-33a3-48c9-bcee-b2c371cbb59d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.218558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] The token cache is being invalidate because project d8b10ff069994911bec6662364b66998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:37:33.218834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.219528 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.220996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1247/2485] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:37:33 2022] DELETE /identity/v3/projects/d8b10ff069994911bec6662364b66998 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:33.225461 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:33.227140 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:33.240499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02537d6-fea5-4eef-8b6d-c31c0ad7005b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:33.274744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:33.300366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a868c-57a2-4885-ba31-9899f3c31e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:33.300998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a868c-57a2-4885-ba31-9899f3c31e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:33.301274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a868c-57a2-4885-ba31-9899f3c31e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:33.301561 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a868c-57a2-4885-ba31-9899f3c31e9b admin admin] PATH_INFO: `/v3/users/b8078fd6bea34d2bae61ab07214076e3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:33.302752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a868c-57a2-4885-ba31-9899f3c31e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8078fd6bea34d2bae61ab07214076e3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:33.304665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a868c-57a2-4885-ba31-9899f3c31e9b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:33.351874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-957a868c-57a2-4885-ba31-9899f3c31e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8078fd6bea34d2bae61ab07214076e3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.360318 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-957a868c-57a2-4885-ba31-9899f3c31e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8078fd6bea34d2bae61ab07214076e3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.361011 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-957a868c-57a2-4885-ba31-9899f3c31e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8078fd6bea34d2bae61ab07214076e3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.367980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-957a868c-57a2-4885-ba31-9899f3c31e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8078fd6bea34d2bae61ab07214076e3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.381267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-957a868c-57a2-4885-ba31-9899f3c31e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8078fd6bea34d2bae61ab07214076e3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.387818 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1239/2486] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:37:33 2022] DELETE /identity/v3/users/b8078fd6bea34d2bae61ab07214076e3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:33.709800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:33.711282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:33.719448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:33.726861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a5e18-33a3-48c9-bcee-b2c371cbb59d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:33.786287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0f350-087b-4714-a08b-3337d408cb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:33.786922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f350-087b-4714-a08b-3337d408cb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:33.787198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f350-087b-4714-a08b-3337d408cb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:33.787462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f350-087b-4714-a08b-3337d408cb9f admin admin] PATH_INFO: `/v3/projects/34f72fb800ed41a583b0d414be302e56` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:33.788692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0f350-087b-4714-a08b-3337d408cb9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f72fb800ed41a583b0d414be302e56)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:33.790690 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0f350-087b-4714-a08b-3337d408cb9f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:33.839912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-25c0f350-087b-4714-a08b-3337d408cb9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f72fb800ed41a583b0d414be302e56', 'request_id': 'req-25c0f350-087b-4714-a08b-3337d408cb9f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.851010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-25c0f350-087b-4714-a08b-3337d408cb9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f72fb800ed41a583b0d414be302e56', 'request_id': 'req-25c0f350-087b-4714-a08b-3337d408cb9f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.857858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-25c0f350-087b-4714-a08b-3337d408cb9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f72fb800ed41a583b0d414be302e56', 'request_id': 'req-25c0f350-087b-4714-a08b-3337d408cb9f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.869659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-25c0f350-087b-4714-a08b-3337d408cb9f admin admin] The token cache is being invalidate because project 34f72fb800ed41a583b0d414be302e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:37:33.869941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-25c0f350-087b-4714-a08b-3337d408cb9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.870658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-25c0f350-087b-4714-a08b-3337d408cb9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:33.872257 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1248/2487] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:37:33 2022] DELETE /identity/v3/projects/34f72fb800ed41a583b0d414be302e56 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:33.878430 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98dccd-b1b6-43b6-b85f-435577229fd0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:33.878927 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98dccd-b1b6-43b6-b85f-435577229fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:33.878927 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98dccd-b1b6-43b6-b85f-435577229fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:33.899427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9f98dccd-b1b6-43b6-b85f-435577229fd0 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:33.900557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9f98dccd-b1b6-43b6-b85f-435577229fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:33.961214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1240/2488] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:33.970293 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e81e7-53de-4bb3-b2f3-139c950cb585 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:33.970544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e81e7-53de-4bb3-b2f3-139c950cb585 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:33.970813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e81e7-53de-4bb3-b2f3-139c950cb585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:33.990780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-528e81e7-53de-4bb3-b2f3-139c950cb585 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:33.992040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-528e81e7-53de-4bb3-b2f3-139c950cb585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:34.000819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1249/2489] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:34.005338 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98dccd-b1b6-43b6-b85f-435577229fd0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:34.014955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98dccd-b1b6-43b6-b85f-435577229fd0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:34.019676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebc0b3-60f6-4209-bc2a-bbde83bf5b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:34.020279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebc0b3-60f6-4209-bc2a-bbde83bf5b03 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.020523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebc0b3-60f6-4209-bc2a-bbde83bf5b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.020824 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebc0b3-60f6-4209-bc2a-bbde83bf5b03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.021545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebc0b3-60f6-4209-bc2a-bbde83bf5b03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:37:34.021918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebc0b3-60f6-4209-bc2a-bbde83bf5b03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:34.023537 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebc0b3-60f6-4209-bc2a-bbde83bf5b03 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:34.035967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1241/2490] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:37:34.040742 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e81e7-53de-4bb3-b2f3-139c950cb585 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:34.050648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e81e7-53de-4bb3-b2f3-139c950cb585 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:34.055769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f628e2-35f3-490e-a174-1e8158091ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:34.056363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f628e2-35f3-490e-a174-1e8158091ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.056645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f628e2-35f3-490e-a174-1e8158091ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.056911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f628e2-35f3-490e-a174-1e8158091ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.057581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f628e2-35f3-490e-a174-1e8158091ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:34.059608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f628e2-35f3-490e-a174-1e8158091ad1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:34.092282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1250/2491] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:37:34 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:34.097164 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebc0b3-60f6-4209-bc2a-bbde83bf5b03 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:34.111849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebc0b3-60f6-4209-bc2a-bbde83bf5b03 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:34.157698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:34.174597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb03c7-3590-4711-9f4e-c35f92e0a26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:34.175205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb03c7-3590-4711-9f4e-c35f92e0a26e admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.175485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb03c7-3590-4711-9f4e-c35f92e0a26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.175758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb03c7-3590-4711-9f4e-c35f92e0a26e admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.176393 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb03c7-3590-4711-9f4e-c35f92e0a26e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:34.178344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb03c7-3590-4711-9f4e-c35f92e0a26e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:34.202256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1242/2492] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:37:34 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:37:34.206537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f628e2-35f3-490e-a174-1e8158091ad1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:34.217554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f628e2-35f3-490e-a174-1e8158091ad1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:34.222760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e6e7b-4a3a-4281-8da6-e986adecf2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:34.223331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e6e7b-4a3a-4281-8da6-e986adecf2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.223611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e6e7b-4a3a-4281-8da6-e986adecf2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.223881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e6e7b-4a3a-4281-8da6-e986adecf2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.224545 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e6e7b-4a3a-4281-8da6-e986adecf2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:34.226189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e6e7b-4a3a-4281-8da6-e986adecf2b0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:34.236127 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1251/2493] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:37:34.240136 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb03c7-3590-4711-9f4e-c35f92e0a26e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:34.250664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb03c7-3590-4711-9f4e-c35f92e0a26e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:34.256320 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee1a7e-14e0-4836-ac0f-230286314844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:34.256942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee1a7e-14e0-4836-ac0f-230286314844 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.257251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee1a7e-14e0-4836-ac0f-230286314844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.257521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee1a7e-14e0-4836-ac0f-230286314844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.258210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee1a7e-14e0-4836-ac0f-230286314844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:34.259828 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee1a7e-14e0-4836-ac0f-230286314844 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:34.270771 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1243/2494] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:34.274858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e6e7b-4a3a-4281-8da6-e986adecf2b0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:34.284957 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e6e7b-4a3a-4281-8da6-e986adecf2b0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:34.290413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511e4c0-93a9-4626-a18a-3611ee5d76e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:34.291068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e4c0-93a9-4626-a18a-3611ee5d76e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.291347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e4c0-93a9-4626-a18a-3611ee5d76e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.291622 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e4c0-93a9-4626-a18a-3611ee5d76e5 admin admin] PATH_INFO: `/v3/projects/df176ced53854b679cdbf5aa08e70afb/users/2a6fa5a14e144b9ea12c92eec438a423/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.307471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511e4c0-93a9-4626-a18a-3611ee5d76e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df176ced53854b679cdbf5aa08e70afb, user_id=2a6fa5a14e144b9ea12c92eec438a423, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:34.310155 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511e4c0-93a9-4626-a18a-3611ee5d76e5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:34.328142 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1252/2495] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:34 2022] PUT /identity/v3/projects/df176ced53854b679cdbf5aa08e70afb/users/2a6fa5a14e144b9ea12c92eec438a423/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:34.396359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ed726-6621-4e66-81d1-26855671c5c6 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.396645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ed726-6621-4e66-81d1-26855671c5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.396955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ed726-6621-4e66-81d1-26855671c5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.434030 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ce9ed726-6621-4e66-81d1-26855671c5c6 None None] MFA Rules not processed for user `2a6fa5a14e144b9ea12c92eec438a423`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:34.435156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ce9ed726-6621-4e66-81d1-26855671c5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:34.503993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1244/2496] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:34.509001 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511e4c0-93a9-4626-a18a-3611ee5d76e5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:34.542184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:34.575378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511e4c0-93a9-4626-a18a-3611ee5d76e5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:34.580512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9112121-a8cd-48a7-a8f4-a49418b869de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:34.581110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112121-a8cd-48a7-a8f4-a49418b869de admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.581483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112121-a8cd-48a7-a8f4-a49418b869de admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.581748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112121-a8cd-48a7-a8f4-a49418b869de admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.582371 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9112121-a8cd-48a7-a8f4-a49418b869de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:34.584261 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9112121-a8cd-48a7-a8f4-a49418b869de admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:34.614010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1253/2497] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:37:34 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:34.618339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ed726-6621-4e66-81d1-26855671c5c6 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:34.631513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ed726-6621-4e66-81d1-26855671c5c6 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:34.698885 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c7a53-e601-4b7a-a000-7bbfc79ca985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:34.699411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c7a53-e601-4b7a-a000-7bbfc79ca985 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.699691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c7a53-e601-4b7a-a000-7bbfc79ca985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.699964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c7a53-e601-4b7a-a000-7bbfc79ca985 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.700860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c7a53-e601-4b7a-a000-7bbfc79ca985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:34.703486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c7a53-e601-4b7a-a000-7bbfc79ca985 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:34.730204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1245/2498] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:37:34 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:34.734428 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9112121-a8cd-48a7-a8f4-a49418b869de admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:34.745113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9112121-a8cd-48a7-a8f4-a49418b869de admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:34.749825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d20d1-0d7c-4227-9577-f7fe3a2f7b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:34.750505 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d20d1-0d7c-4227-9577-f7fe3a2f7b9b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.750851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d20d1-0d7c-4227-9577-f7fe3a2f7b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.751175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d20d1-0d7c-4227-9577-f7fe3a2f7b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.751884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d20d1-0d7c-4227-9577-f7fe3a2f7b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:34.754568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d20d1-0d7c-4227-9577-f7fe3a2f7b9b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:34.767876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1254/2499] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:34.769155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c7a53-e601-4b7a-a000-7bbfc79ca985 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:34.779123 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c7a53-e601-4b7a-a000-7bbfc79ca985 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:34.783728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff1b62-cfb8-4a63-a6ea-1d6c67f4bee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:34.784382 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff1b62-cfb8-4a63-a6ea-1d6c67f4bee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.784653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff1b62-cfb8-4a63-a6ea-1d6c67f4bee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.784921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff1b62-cfb8-4a63-a6ea-1d6c67f4bee6 admin admin] PATH_INFO: `/v3/projects/c608375a16954dbdbdcc43d1430fd950/users/f7b7379fa7864059aee928af60bd9ec6/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.800143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff1b62-cfb8-4a63-a6ea-1d6c67f4bee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c608375a16954dbdbdcc43d1430fd950, user_id=f7b7379fa7864059aee928af60bd9ec6, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:34.802801 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff1b62-cfb8-4a63-a6ea-1d6c67f4bee6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:34.817861 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1246/2500] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:34 2022] PUT /identity/v3/projects/c608375a16954dbdbdcc43d1430fd950/users/f7b7379fa7864059aee928af60bd9ec6/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:34.822061 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d20d1-0d7c-4227-9577-f7fe3a2f7b9b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:34.832372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d20d1-0d7c-4227-9577-f7fe3a2f7b9b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:34.893229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341c601-43b4-4852-82c9-0d3d9fda0e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:34.893835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341c601-43b4-4852-82c9-0d3d9fda0e21 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.894124 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341c601-43b4-4852-82c9-0d3d9fda0e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.894395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341c601-43b4-4852-82c9-0d3d9fda0e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.895028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341c601-43b4-4852-82c9-0d3d9fda0e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:34.896669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341c601-43b4-4852-82c9-0d3d9fda0e21 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:34.910573 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1255/2501] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:34.911815 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff1b62-cfb8-4a63-a6ea-1d6c67f4bee6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:34.923287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff1b62-cfb8-4a63-a6ea-1d6c67f4bee6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:34.928350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96e4c2-31c2-4078-9079-d83e6585061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:34.929111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96e4c2-31c2-4078-9079-d83e6585061d admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.929484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96e4c2-31c2-4078-9079-d83e6585061d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.929822 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96e4c2-31c2-4078-9079-d83e6585061d admin admin] PATH_INFO: `/v3/projects/c608375a16954dbdbdcc43d1430fd950/users/f7b7379fa7864059aee928af60bd9ec6/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.931822 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96e4c2-31c2-4078-9079-d83e6585061d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c608375a16954dbdbdcc43d1430fd950, user_id=f7b7379fa7864059aee928af60bd9ec6, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:34.934588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96e4c2-31c2-4078-9079-d83e6585061d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:34.945760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1247/2502] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:34 2022] PUT /identity/v3/projects/c608375a16954dbdbdcc43d1430fd950/users/f7b7379fa7864059aee928af60bd9ec6/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:34.956347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad80617-d80c-4098-89e1-535404faff49 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:34.956632 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad80617-d80c-4098-89e1-535404faff49 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:34.956899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad80617-d80c-4098-89e1-535404faff49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:34.986110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cad80617-d80c-4098-89e1-535404faff49 None None] MFA Rules not processed for user `f7b7379fa7864059aee928af60bd9ec6`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:34.987325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cad80617-d80c-4098-89e1-535404faff49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:35.060143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1256/2503] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:35.071562 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96e4c2-31c2-4078-9079-d83e6585061d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:35.073392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fd96e4c2-31c2-4078-9079-d83e6585061d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:35.080122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:35.140825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96e4c2-31c2-4078-9079-d83e6585061d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:35.144807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fd022-9e40-4e6d-92fe-32f6671938f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:35.145524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fd022-9e40-4e6d-92fe-32f6671938f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:35.145876 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fd022-9e40-4e6d-92fe-32f6671938f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:35.146148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fd022-9e40-4e6d-92fe-32f6671938f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:35.156132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fd022-9e40-4e6d-92fe-32f6671938f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:35.157701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fd022-9e40-4e6d-92fe-32f6671938f3 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:35.173020 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1248/2504] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:35.578522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad80617-d80c-4098-89e1-535404faff49 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:35.613359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:35.651169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad80617-d80c-4098-89e1-535404faff49 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:35.656316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:35.656945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:35.657266 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:35.657539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] PATH_INFO: `/v3/users/2a6fa5a14e144b9ea12c92eec438a423` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:35.658735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a6fa5a14e144b9ea12c92eec438a423)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:35.660659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:35.708884 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a6fa5a14e144b9ea12c92eec438a423'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:35.717244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a6fa5a14e144b9ea12c92eec438a423'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:35.717839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a6fa5a14e144b9ea12c92eec438a423'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:35.725190 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a6fa5a14e144b9ea12c92eec438a423'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:35.737353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a6fa5a14e144b9ea12c92eec438a423'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:35.743687 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1257/2505] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:37:35 2022] DELETE /identity/v3/users/2a6fa5a14e144b9ea12c92eec438a423 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:35.748468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14a593-112e-4ea3-af41-c74e3ff70e08 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:35.748760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14a593-112e-4ea3-af41-c74e3ff70e08 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:35.749091 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14a593-112e-4ea3-af41-c74e3ff70e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:35.769624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fb14a593-112e-4ea3-af41-c74e3ff70e08 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:35.771616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fb14a593-112e-4ea3-af41-c74e3ff70e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:35.832722 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1249/2506] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:37:35.843153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:35.844622 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:35.902491 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d86735-0cc3-467f-9288-ec5ade03daa3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:35.906490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bc147-57f6-43fd-8d06-f627163de5ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:35.907112 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bc147-57f6-43fd-8d06-f627163de5ec service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:35.907412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bc147-57f6-43fd-8d06-f627163de5ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:35.907677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bc147-57f6-43fd-8d06-f627163de5ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:35.917502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bc147-57f6-43fd-8d06-f627163de5ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:35.918983 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bc147-57f6-43fd-8d06-f627163de5ec service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:35.941714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1258/2507] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:36.229596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14a593-112e-4ea3-af41-c74e3ff70e08 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:36.231814 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fb14a593-112e-4ea3-af41-c74e3ff70e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:36.244723 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14a593-112e-4ea3-af41-c74e3ff70e08 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:36.253230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:36.253857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:36.254163 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:36.254434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] PATH_INFO: `/v3/projects/df176ced53854b679cdbf5aa08e70afb` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:36.255631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df176ced53854b679cdbf5aa08e70afb)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:36.257624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:36.306913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df176ced53854b679cdbf5aa08e70afb', 'request_id': 'req-613fe158-a02a-4075-b114-a2bb890a4742'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.315410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df176ced53854b679cdbf5aa08e70afb', 'request_id': 'req-613fe158-a02a-4075-b114-a2bb890a4742'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.322268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df176ced53854b679cdbf5aa08e70afb', 'request_id': 'req-613fe158-a02a-4075-b114-a2bb890a4742'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.334736 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] The token cache is being invalidate because project df176ced53854b679cdbf5aa08e70afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:37:36.334736 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.335380 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.337049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1250/2508] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:37:36 2022] DELETE /identity/v3/projects/df176ced53854b679cdbf5aa08e70afb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:36.341743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bc147-57f6-43fd-8d06-f627163de5ec service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:36.351863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bc147-57f6-43fd-8d06-f627163de5ec service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:36.397800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:36.412705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fda5f-b8f9-4df8-a1d7-14491903723f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:36.413377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fda5f-b8f9-4df8-a1d7-14491903723f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:36.413652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fda5f-b8f9-4df8-a1d7-14491903723f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:36.413925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fda5f-b8f9-4df8-a1d7-14491903723f admin admin] PATH_INFO: `/v3/users/f7b7379fa7864059aee928af60bd9ec6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:36.415068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fda5f-b8f9-4df8-a1d7-14491903723f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7b7379fa7864059aee928af60bd9ec6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:36.417032 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fda5f-b8f9-4df8-a1d7-14491903723f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:36.465053 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-909fda5f-b8f9-4df8-a1d7-14491903723f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7b7379fa7864059aee928af60bd9ec6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.483369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-909fda5f-b8f9-4df8-a1d7-14491903723f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7b7379fa7864059aee928af60bd9ec6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.484265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-909fda5f-b8f9-4df8-a1d7-14491903723f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7b7379fa7864059aee928af60bd9ec6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.492192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-909fda5f-b8f9-4df8-a1d7-14491903723f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7b7379fa7864059aee928af60bd9ec6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.505049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-909fda5f-b8f9-4df8-a1d7-14491903723f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7b7379fa7864059aee928af60bd9ec6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.511389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1259/2509] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:37:36 2022] DELETE /identity/v3/users/f7b7379fa7864059aee928af60bd9ec6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:36.797579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:36.799162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:36.819650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fe158-a02a-4075-b114-a2bb890a4742 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:36.861955 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:36.884465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb90ae7-1819-4601-b799-58989f5a8c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:36.885170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb90ae7-1819-4601-b799-58989f5a8c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:36.885777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb90ae7-1819-4601-b799-58989f5a8c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:36.886246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb90ae7-1819-4601-b799-58989f5a8c01 admin admin] PATH_INFO: `/v3/projects/c608375a16954dbdbdcc43d1430fd950` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:36.888000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb90ae7-1819-4601-b799-58989f5a8c01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c608375a16954dbdbdcc43d1430fd950)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:36.889981 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb90ae7-1819-4601-b799-58989f5a8c01 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:36.940665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2eb90ae7-1819-4601-b799-58989f5a8c01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c608375a16954dbdbdcc43d1430fd950', 'request_id': 'req-2eb90ae7-1819-4601-b799-58989f5a8c01'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.948899 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2eb90ae7-1819-4601-b799-58989f5a8c01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c608375a16954dbdbdcc43d1430fd950', 'request_id': 'req-2eb90ae7-1819-4601-b799-58989f5a8c01'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.955968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2eb90ae7-1819-4601-b799-58989f5a8c01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c608375a16954dbdbdcc43d1430fd950', 'request_id': 'req-2eb90ae7-1819-4601-b799-58989f5a8c01'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.968273 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2eb90ae7-1819-4601-b799-58989f5a8c01 admin admin] The token cache is being invalidate because project c608375a16954dbdbdcc43d1430fd950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:37:36.968719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2eb90ae7-1819-4601-b799-58989f5a8c01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.969443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2eb90ae7-1819-4601-b799-58989f5a8c01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:36.971102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1251/2510] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:37:36 2022] DELETE /identity/v3/projects/c608375a16954dbdbdcc43d1430fd950 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:36.976172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843f1eb-8ec4-494b-9f29-35aa2d1d0759 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:36.976638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843f1eb-8ec4-494b-9f29-35aa2d1d0759 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:36.976751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843f1eb-8ec4-494b-9f29-35aa2d1d0759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:36.996643 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4843f1eb-8ec4-494b-9f29-35aa2d1d0759 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:36.997927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4843f1eb-8ec4-494b-9f29-35aa2d1d0759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:37.059134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1260/2511] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:37.068376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7ddca-1cb9-431d-8c6f-ac4e83ce15fa None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.068650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7ddca-1cb9-431d-8c6f-ac4e83ce15fa None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.068923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7ddca-1cb9-431d-8c6f-ac4e83ce15fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.090176 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-67d7ddca-1cb9-431d-8c6f-ac4e83ce15fa None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:37.091744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-67d7ddca-1cb9-431d-8c6f-ac4e83ce15fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:37.100719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1252/2512] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:37.105058 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843f1eb-8ec4-494b-9f29-35aa2d1d0759 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:37.114957 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843f1eb-8ec4-494b-9f29-35aa2d1d0759 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:37.120110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbd5a0-7a0f-4b1e-bc41-022e6521ed20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:37.121753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbd5a0-7a0f-4b1e-bc41-022e6521ed20 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.122090 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbd5a0-7a0f-4b1e-bc41-022e6521ed20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.122367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbd5a0-7a0f-4b1e-bc41-022e6521ed20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.123057 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbd5a0-7a0f-4b1e-bc41-022e6521ed20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:37:37.123501 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbd5a0-7a0f-4b1e-bc41-022e6521ed20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:37.125216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbd5a0-7a0f-4b1e-bc41-022e6521ed20 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:37.137661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1261/2513] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:37:37.142168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7ddca-1cb9-431d-8c6f-ac4e83ce15fa None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:37.151789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7ddca-1cb9-431d-8c6f-ac4e83ce15fa None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:37.157384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd3115-01e3-4677-b51c-65318eef6ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:37.158003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd3115-01e3-4677-b51c-65318eef6ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.158330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd3115-01e3-4677-b51c-65318eef6ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.158602 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd3115-01e3-4677-b51c-65318eef6ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.159245 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd3115-01e3-4677-b51c-65318eef6ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:37.161167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd3115-01e3-4677-b51c-65318eef6ea6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:37.194256 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1253/2514] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:37:37 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:37:37.195600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbd5a0-7a0f-4b1e-bc41-022e6521ed20 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:37.206268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbd5a0-7a0f-4b1e-bc41-022e6521ed20 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:37.255599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:37.267560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3552f9-39f7-458f-9a03-df90c003dba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:37.268180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3552f9-39f7-458f-9a03-df90c003dba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.268460 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3552f9-39f7-458f-9a03-df90c003dba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.268733 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3552f9-39f7-458f-9a03-df90c003dba6 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.269429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3552f9-39f7-458f-9a03-df90c003dba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:37.271390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3552f9-39f7-458f-9a03-df90c003dba6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:37.294095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1262/2515] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:37:37 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:37.298309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd3115-01e3-4677-b51c-65318eef6ea6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:37.308912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd3115-01e3-4677-b51c-65318eef6ea6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:37.313817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669abc93-b2ee-4eac-a0fe-0c439bbf969b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:37.314342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669abc93-b2ee-4eac-a0fe-0c439bbf969b admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.314618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669abc93-b2ee-4eac-a0fe-0c439bbf969b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.314917 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669abc93-b2ee-4eac-a0fe-0c439bbf969b admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.315595 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669abc93-b2ee-4eac-a0fe-0c439bbf969b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:37.317217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669abc93-b2ee-4eac-a0fe-0c439bbf969b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:37.327481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1254/2516] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:37.331580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3552f9-39f7-458f-9a03-df90c003dba6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:37.341940 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3552f9-39f7-458f-9a03-df90c003dba6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:37.346727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb87caf-2214-4408-9940-87b2627e0330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:37.347323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb87caf-2214-4408-9940-87b2627e0330 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.347603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb87caf-2214-4408-9940-87b2627e0330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.347877 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb87caf-2214-4408-9940-87b2627e0330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.348521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb87caf-2214-4408-9940-87b2627e0330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:37.350152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb87caf-2214-4408-9940-87b2627e0330 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:37.360769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1263/2517] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:37:37.365219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669abc93-b2ee-4eac-a0fe-0c439bbf969b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:37.380631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669abc93-b2ee-4eac-a0fe-0c439bbf969b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:37.389192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740a274-c816-4e83-a0ee-60c919605ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:37.389851 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740a274-c816-4e83-a0ee-60c919605ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.390127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740a274-c816-4e83-a0ee-60c919605ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.390399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740a274-c816-4e83-a0ee-60c919605ac5 admin admin] PATH_INFO: `/v3/projects/98d0823fce8c41099e47fa1ebb634877/users/8c6b2a9c34fc42b4b59fcdcaf5f25e17/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.406485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740a274-c816-4e83-a0ee-60c919605ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d0823fce8c41099e47fa1ebb634877, user_id=8c6b2a9c34fc42b4b59fcdcaf5f25e17, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:37.409178 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740a274-c816-4e83-a0ee-60c919605ac5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:37.426915 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1255/2518] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:37 2022] PUT /identity/v3/projects/98d0823fce8c41099e47fa1ebb634877/users/8c6b2a9c34fc42b4b59fcdcaf5f25e17/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:37.437617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a39a42-0090-47bc-858c-fc5f9a78199f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.437905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a39a42-0090-47bc-858c-fc5f9a78199f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.438178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a39a42-0090-47bc-858c-fc5f9a78199f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.473829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-72a39a42-0090-47bc-858c-fc5f9a78199f None None] MFA Rules not processed for user `8c6b2a9c34fc42b4b59fcdcaf5f25e17`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:37.475599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-72a39a42-0090-47bc-858c-fc5f9a78199f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:37.545596 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1264/2519] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:37.550788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740a274-c816-4e83-a0ee-60c919605ac5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:37.623309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:37.627528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740a274-c816-4e83-a0ee-60c919605ac5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:37.633829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed8f0c-c690-46b8-91e3-ec4439d84e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:37.634540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed8f0c-c690-46b8-91e3-ec4439d84e1b admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.635628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed8f0c-c690-46b8-91e3-ec4439d84e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.636442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed8f0c-c690-46b8-91e3-ec4439d84e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.637100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed8f0c-c690-46b8-91e3-ec4439d84e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:37.639316 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed8f0c-c690-46b8-91e3-ec4439d84e1b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:37.669719 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1256/2520] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:37:37 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:37.674168 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a39a42-0090-47bc-858c-fc5f9a78199f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:37.684843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a39a42-0090-47bc-858c-fc5f9a78199f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:37.740698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ca526-b68a-4298-9385-34f05db0d6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:37.741336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ca526-b68a-4298-9385-34f05db0d6af admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.741615 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ca526-b68a-4298-9385-34f05db0d6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.741883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ca526-b68a-4298-9385-34f05db0d6af admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.742522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ca526-b68a-4298-9385-34f05db0d6af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:37.744448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ca526-b68a-4298-9385-34f05db0d6af admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:37.767569 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1265/2521] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:37:37 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:37.771657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed8f0c-c690-46b8-91e3-ec4439d84e1b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:37.782990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed8f0c-c690-46b8-91e3-ec4439d84e1b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:37.787995 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b5896-305e-4508-9148-8be70d125c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:37.788600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b5896-305e-4508-9148-8be70d125c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.788889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b5896-305e-4508-9148-8be70d125c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.789194 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b5896-305e-4508-9148-8be70d125c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.789843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b5896-305e-4508-9148-8be70d125c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:37.792218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b5896-305e-4508-9148-8be70d125c5b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:37.803788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1257/2522] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:37.807920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ca526-b68a-4298-9385-34f05db0d6af admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:37.818056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ca526-b68a-4298-9385-34f05db0d6af admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:37.822948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db3aaf-efde-476d-9683-db6626bfd573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:37.823654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db3aaf-efde-476d-9683-db6626bfd573 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.823929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db3aaf-efde-476d-9683-db6626bfd573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.824195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db3aaf-efde-476d-9683-db6626bfd573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.824840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db3aaf-efde-476d-9683-db6626bfd573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:37.826494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db3aaf-efde-476d-9683-db6626bfd573 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:37.838645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1266/2523] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:37.841966 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b5896-305e-4508-9148-8be70d125c5b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:37.852407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b5896-305e-4508-9148-8be70d125c5b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:37.857708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba9778-5d0c-446e-abbc-d677ff09ccd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:37.858367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba9778-5d0c-446e-abbc-d677ff09ccd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.858645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba9778-5d0c-446e-abbc-d677ff09ccd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.858914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba9778-5d0c-446e-abbc-d677ff09ccd0 admin admin] PATH_INFO: `/v3/projects/1e368f187cda40c0a826b875c35d482e/users/fd7153bc30c24e779f237ca61df0bde9/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.874552 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba9778-5d0c-446e-abbc-d677ff09ccd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e368f187cda40c0a826b875c35d482e, user_id=fd7153bc30c24e779f237ca61df0bde9, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:37.877330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba9778-5d0c-446e-abbc-d677ff09ccd0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:37.893691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1258/2524] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:37 2022] PUT /identity/v3/projects/1e368f187cda40c0a826b875c35d482e/users/fd7153bc30c24e779f237ca61df0bde9/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:37.904323 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abda714-b0ea-4dab-8469-4eb1e1c37b70 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:37.904838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abda714-b0ea-4dab-8469-4eb1e1c37b70 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:37.905225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abda714-b0ea-4dab-8469-4eb1e1c37b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:37.942720 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2abda714-b0ea-4dab-8469-4eb1e1c37b70 None None] MFA Rules not processed for user `fd7153bc30c24e779f237ca61df0bde9`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:37.943991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2abda714-b0ea-4dab-8469-4eb1e1c37b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:37.975383 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:38.019113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1267/2525] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:38.024321 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba9778-5d0c-446e-abbc-d677ff09ccd0 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:38.086144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba9778-5d0c-446e-abbc-d677ff09ccd0 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:38.091425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bda96-a1e0-425e-9e78-d1966b3887b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:38.091994 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bda96-a1e0-425e-9e78-d1966b3887b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:38.092268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bda96-a1e0-425e-9e78-d1966b3887b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:38.092533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bda96-a1e0-425e-9e78-d1966b3887b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:38.093218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bda96-a1e0-425e-9e78-d1966b3887b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:38.095112 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bda96-a1e0-425e-9e78-d1966b3887b2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:38.122411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1259/2526] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:37:38 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:38.126554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abda714-b0ea-4dab-8469-4eb1e1c37b70 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:38.138064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abda714-b0ea-4dab-8469-4eb1e1c37b70 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:38.193457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1daf74c-6a4c-48ef-9459-6ee68d581258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:38.194080 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daf74c-6a4c-48ef-9459-6ee68d581258 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:38.194348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daf74c-6a4c-48ef-9459-6ee68d581258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:38.194617 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daf74c-6a4c-48ef-9459-6ee68d581258 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:38.195267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1daf74c-6a4c-48ef-9459-6ee68d581258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:38.197200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1daf74c-6a4c-48ef-9459-6ee68d581258 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:38.220514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1268/2527] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:37:38 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:37:38.225004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bda96-a1e0-425e-9e78-d1966b3887b2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:38.235619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bda96-a1e0-425e-9e78-d1966b3887b2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:38.240175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0a34b-765a-4620-b3ea-fffb11f1fded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:38.240770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0a34b-765a-4620-b3ea-fffb11f1fded admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:38.241039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0a34b-765a-4620-b3ea-fffb11f1fded admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:38.241336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0a34b-765a-4620-b3ea-fffb11f1fded admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:38.241971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0a34b-765a-4620-b3ea-fffb11f1fded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:38.243567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0a34b-765a-4620-b3ea-fffb11f1fded admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:38.257674 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1260/2528] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:38.258918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1daf74c-6a4c-48ef-9459-6ee68d581258 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:38.270412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1daf74c-6a4c-48ef-9459-6ee68d581258 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:38.276447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34c085-319a-4229-8008-91e2da67aa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:38.277180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34c085-319a-4229-8008-91e2da67aa57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:38.277437 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34c085-319a-4229-8008-91e2da67aa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:38.277708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34c085-319a-4229-8008-91e2da67aa57 admin admin] PATH_INFO: `/v3/projects/2fa411ecc0c041f68c65a4ef7df518e1/users/92df8d2f8398481f945372aa5e5cd286/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:38.294893 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34c085-319a-4229-8008-91e2da67aa57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa411ecc0c041f68c65a4ef7df518e1, user_id=92df8d2f8398481f945372aa5e5cd286, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:38.297670 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34c085-319a-4229-8008-91e2da67aa57 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:38.312685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1269/2529] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:38 2022] PUT /identity/v3/projects/2fa411ecc0c041f68c65a4ef7df518e1/users/92df8d2f8398481f945372aa5e5cd286/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:38.316473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0a34b-765a-4620-b3ea-fffb11f1fded admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:38.333710 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0a34b-765a-4620-b3ea-fffb11f1fded admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:38.394815 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af2a30-f60b-41ed-9984-b1114a0893f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:38.395344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af2a30-f60b-41ed-9984-b1114a0893f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:38.395630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af2a30-f60b-41ed-9984-b1114a0893f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:38.395935 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af2a30-f60b-41ed-9984-b1114a0893f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:38.396585 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af2a30-f60b-41ed-9984-b1114a0893f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:38.398213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af2a30-f60b-41ed-9984-b1114a0893f6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:38.408531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1261/2530] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:37:38.412907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34c085-319a-4229-8008-91e2da67aa57 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:38.423504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34c085-319a-4229-8008-91e2da67aa57 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:38.428118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26cfc9-4c58-4f5e-9ee8-9d7c69bc8797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:38.428777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26cfc9-4c58-4f5e-9ee8-9d7c69bc8797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:38.429107 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26cfc9-4c58-4f5e-9ee8-9d7c69bc8797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:38.429377 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26cfc9-4c58-4f5e-9ee8-9d7c69bc8797 admin admin] PATH_INFO: `/v3/projects/2fa411ecc0c041f68c65a4ef7df518e1/users/92df8d2f8398481f945372aa5e5cd286/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:38.431305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26cfc9-4c58-4f5e-9ee8-9d7c69bc8797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa411ecc0c041f68c65a4ef7df518e1, user_id=92df8d2f8398481f945372aa5e5cd286, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:38.433952 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26cfc9-4c58-4f5e-9ee8-9d7c69bc8797 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:38.444367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1270/2531] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:38 2022] PUT /identity/v3/projects/2fa411ecc0c041f68c65a4ef7df518e1/users/92df8d2f8398481f945372aa5e5cd286/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:38.455810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fe38b-e272-4b5e-b62c-11c2d0fab119 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:38.456093 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fe38b-e272-4b5e-b62c-11c2d0fab119 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:38.456366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fe38b-e272-4b5e-b62c-11c2d0fab119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:38.486397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-013fe38b-e272-4b5e-b62c-11c2d0fab119 None None] MFA Rules not processed for user `92df8d2f8398481f945372aa5e5cd286`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:38.487602 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-013fe38b-e272-4b5e-b62c-11c2d0fab119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:38.492658 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:38.560200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1262/2532] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:38.572333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26cfc9-4c58-4f5e-9ee8-9d7c69bc8797 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:38.574341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1b26cfc9-4c58-4f5e-9ee8-9d7c69bc8797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:38.631906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26cfc9-4c58-4f5e-9ee8-9d7c69bc8797 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:38.636661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1a51b-e895-4b16-8ffc-a305071c2786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:38.637326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1a51b-e895-4b16-8ffc-a305071c2786 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:38.637601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1a51b-e895-4b16-8ffc-a305071c2786 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:38.637874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1a51b-e895-4b16-8ffc-a305071c2786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:38.673411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:38.699507 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1a51b-e895-4b16-8ffc-a305071c2786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:38.700984 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1a51b-e895-4b16-8ffc-a305071c2786 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:38.710121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1271/2533] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:40.234447 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fe38b-e272-4b5e-b62c-11c2d0fab119 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:40.236049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-013fe38b-e272-4b5e-b62c-11c2d0fab119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:40.247998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fe38b-e272-4b5e-b62c-11c2d0fab119 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:40.251774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86836bad-ecfb-4398-9171-556fc4ab97d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:40.252378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86836bad-ecfb-4398-9171-556fc4ab97d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:40.252648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86836bad-ecfb-4398-9171-556fc4ab97d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:40.252917 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86836bad-ecfb-4398-9171-556fc4ab97d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:40.321413 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86836bad-ecfb-4398-9171-556fc4ab97d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:40.322769 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86836bad-ecfb-4398-9171-556fc4ab97d0 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:40.332523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1263/2534] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:40.892511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1a51b-e895-4b16-8ffc-a305071c2786 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:40.901149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1a51b-e895-4b16-8ffc-a305071c2786 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:40.905490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e490cc-7055-47bf-a339-57b83db895c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:40.906639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e490cc-7055-47bf-a339-57b83db895c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:40.906639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e490cc-7055-47bf-a339-57b83db895c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:40.906799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e490cc-7055-47bf-a339-57b83db895c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:40.920476 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e490cc-7055-47bf-a339-57b83db895c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:40.921962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e490cc-7055-47bf-a339-57b83db895c1 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:40.932392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1272/2535] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:45.323343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6061039-87cc-4c32-8cb8-eeb223d5e447 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:45.323836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6061039-87cc-4c32-8cb8-eeb223d5e447 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:45.323928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6061039-87cc-4c32-8cb8-eeb223d5e447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:45.355587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e6061039-87cc-4c32-8cb8-eeb223d5e447 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:45.357053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e6061039-87cc-4c32-8cb8-eeb223d5e447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:45.426196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1264/2536] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:37:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:45.476980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e490cc-7055-47bf-a339-57b83db895c1 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:45.478584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-31e490cc-7055-47bf-a339-57b83db895c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:45.549809 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e490cc-7055-47bf-a339-57b83db895c1 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:45.556029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b6e08-a93b-4c57-9331-0d1b04c05591 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:45.556635 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b6e08-a93b-4c57-9331-0d1b04c05591 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:45.556905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b6e08-a93b-4c57-9331-0d1b04c05591 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:45.557333 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b6e08-a93b-4c57-9331-0d1b04c05591 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:45.566271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b6e08-a93b-4c57-9331-0d1b04c05591 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:45.567840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b6e08-a93b-4c57-9331-0d1b04c05591 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:45.576144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1273/2537] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:37:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:37:45.978105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6061039-87cc-4c32-8cb8-eeb223d5e447 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:46.027758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:46.053201 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6061039-87cc-4c32-8cb8-eeb223d5e447 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:46.058296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:46.058956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:46.059232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:46.059672 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] PATH_INFO: `/v3/users/8c6b2a9c34fc42b4b59fcdcaf5f25e17` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:46.062373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6b2a9c34fc42b4b59fcdcaf5f25e17)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:46.063191 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:46.127062 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6b2a9c34fc42b4b59fcdcaf5f25e17'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:46.139708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6b2a9c34fc42b4b59fcdcaf5f25e17'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:46.140509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6b2a9c34fc42b4b59fcdcaf5f25e17'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:46.150359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6b2a9c34fc42b4b59fcdcaf5f25e17'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:46.164834 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6b2a9c34fc42b4b59fcdcaf5f25e17'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:46.173134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1265/2538] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:37:45 2022] DELETE /identity/v3/users/8c6b2a9c34fc42b4b59fcdcaf5f25e17 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:46.177201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c62740-13e9-4b82-8ea3-1cab1dcdf7ed None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:46.177613 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c62740-13e9-4b82-8ea3-1cab1dcdf7ed None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:46.177986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c62740-13e9-4b82-8ea3-1cab1dcdf7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:46.198694 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-97c62740-13e9-4b82-8ea3-1cab1dcdf7ed None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:46.199952 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-97c62740-13e9-4b82-8ea3-1cab1dcdf7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:46.276654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1274/2539] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:46.286678 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:46.288129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:46.371485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f308eb-cd21-4741-97b3-fac9f833dc68 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:46.375555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9301e5d-c69b-4c1c-bc4b-dd764f8aecb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:46.376266 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9301e5d-c69b-4c1c-bc4b-dd764f8aecb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:46.376605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9301e5d-c69b-4c1c-bc4b-dd764f8aecb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:46.376943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9301e5d-c69b-4c1c-bc4b-dd764f8aecb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:46.387323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9301e5d-c69b-4c1c-bc4b-dd764f8aecb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:46.389217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9301e5d-c69b-4c1c-bc4b-dd764f8aecb5 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:46.400486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1266/2540] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:46.742312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c62740-13e9-4b82-8ea3-1cab1dcdf7ed None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:46.744653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-97c62740-13e9-4b82-8ea3-1cab1dcdf7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:46.762581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c62740-13e9-4b82-8ea3-1cab1dcdf7ed None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:46.767931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:46.769331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:46.769854 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:46.770801 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] PATH_INFO: `/v3/projects/98d0823fce8c41099e47fa1ebb634877` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:46.772998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d0823fce8c41099e47fa1ebb634877)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:46.775121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:46.830768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:46.831754 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d0823fce8c41099e47fa1ebb634877', 'request_id': 'req-465a1ec3-3469-400b-8462-7efed9c535d0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:46.842178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d0823fce8c41099e47fa1ebb634877', 'request_id': 'req-465a1ec3-3469-400b-8462-7efed9c535d0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:46.849052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d0823fce8c41099e47fa1ebb634877', 'request_id': 'req-465a1ec3-3469-400b-8462-7efed9c535d0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:46.861174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] The token cache is being invalidate because project 98d0823fce8c41099e47fa1ebb634877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:37:46.862262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:46.863648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:46.865226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1275/2541] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:37:46 2022] DELETE /identity/v3/projects/98d0823fce8c41099e47fa1ebb634877 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:46.869404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9301e5d-c69b-4c1c-bc4b-dd764f8aecb5 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:46.890607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9301e5d-c69b-4c1c-bc4b-dd764f8aecb5 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:46.961765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:46.962403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:46.962681 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:46.962952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] PATH_INFO: `/v3/users/fd7153bc30c24e779f237ca61df0bde9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:46.964090 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd7153bc30c24e779f237ca61df0bde9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:46.968415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:46.970385 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:47.034779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd7153bc30c24e779f237ca61df0bde9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.047375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd7153bc30c24e779f237ca61df0bde9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.048145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd7153bc30c24e779f237ca61df0bde9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.055957 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd7153bc30c24e779f237ca61df0bde9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.070784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd7153bc30c24e779f237ca61df0bde9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.077032 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1267/2542] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:37:46 2022] DELETE /identity/v3/users/fd7153bc30c24e779f237ca61df0bde9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:47.408121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:47.409852 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:47.426801 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a1ec3-3469-400b-8462-7efed9c535d0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:47.513049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:47.514043 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:47.514320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:47.514589 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] PATH_INFO: `/v3/projects/1e368f187cda40c0a826b875c35d482e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:47.515947 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e368f187cda40c0a826b875c35d482e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:47.517912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:47.597803 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e368f187cda40c0a826b875c35d482e', 'request_id': 'req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.609682 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e368f187cda40c0a826b875c35d482e', 'request_id': 'req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.621419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e368f187cda40c0a826b875c35d482e', 'request_id': 'req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.638417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] The token cache is being invalidate because project 1e368f187cda40c0a826b875c35d482e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:37:47.638752 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.639382 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.643805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1276/2543] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:37:47 2022] DELETE /identity/v3/projects/1e368f187cda40c0a826b875c35d482e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:47.645315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:47.647184 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:47.676413 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a59d60-e78a-4d4e-b1a5-dfa4a61e9e17 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:47.760616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225d93f-add0-4501-97e6-ff67a82d14a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:47.761336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225d93f-add0-4501-97e6-ff67a82d14a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:47.761633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225d93f-add0-4501-97e6-ff67a82d14a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:47.761890 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225d93f-add0-4501-97e6-ff67a82d14a6 admin admin] PATH_INFO: `/v3/users/92df8d2f8398481f945372aa5e5cd286` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:47.763268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225d93f-add0-4501-97e6-ff67a82d14a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92df8d2f8398481f945372aa5e5cd286)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:47.765582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225d93f-add0-4501-97e6-ff67a82d14a6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:47.824229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a225d93f-add0-4501-97e6-ff67a82d14a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92df8d2f8398481f945372aa5e5cd286'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.834040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a225d93f-add0-4501-97e6-ff67a82d14a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92df8d2f8398481f945372aa5e5cd286'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.834708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a225d93f-add0-4501-97e6-ff67a82d14a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92df8d2f8398481f945372aa5e5cd286'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.845506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a225d93f-add0-4501-97e6-ff67a82d14a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92df8d2f8398481f945372aa5e5cd286'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.858388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a225d93f-add0-4501-97e6-ff67a82d14a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92df8d2f8398481f945372aa5e5cd286'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:47.859950 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:47.866116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1268/2544] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:37:47 2022] DELETE /identity/v3/users/92df8d2f8398481f945372aa5e5cd286 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:37:48.205730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:48.207827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:48.228802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a422c-c9e4-46ff-9785-5bf4d3309fa7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:48.305959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d7427-406c-4b10-a1b6-09c42a1d36b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:48.306954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d7427-406c-4b10-a1b6-09c42a1d36b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:48.307237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d7427-406c-4b10-a1b6-09c42a1d36b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:48.307503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d7427-406c-4b10-a1b6-09c42a1d36b8 admin admin] PATH_INFO: `/v3/projects/2fa411ecc0c041f68c65a4ef7df518e1` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:48.308723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d7427-406c-4b10-a1b6-09c42a1d36b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fa411ecc0c041f68c65a4ef7df518e1)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:48.310845 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d7427-406c-4b10-a1b6-09c42a1d36b8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:48.320116 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:48.374826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-344d7427-406c-4b10-a1b6-09c42a1d36b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa411ecc0c041f68c65a4ef7df518e1', 'request_id': 'req-344d7427-406c-4b10-a1b6-09c42a1d36b8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:48.383776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-344d7427-406c-4b10-a1b6-09c42a1d36b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa411ecc0c041f68c65a4ef7df518e1', 'request_id': 'req-344d7427-406c-4b10-a1b6-09c42a1d36b8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:48.390405 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-344d7427-406c-4b10-a1b6-09c42a1d36b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa411ecc0c041f68c65a4ef7df518e1', 'request_id': 'req-344d7427-406c-4b10-a1b6-09c42a1d36b8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:48.402692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-344d7427-406c-4b10-a1b6-09c42a1d36b8 admin admin] The token cache is being invalidate because project 2fa411ecc0c041f68c65a4ef7df518e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:37:48.403005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-344d7427-406c-4b10-a1b6-09c42a1d36b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:48.403668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-344d7427-406c-4b10-a1b6-09c42a1d36b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:37:48.405211 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1277/2545] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:37:48 2022] DELETE /identity/v3/projects/2fa411ecc0c041f68c65a4ef7df518e1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:48.411045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0815e94-06c9-430f-9f6a-4e328c9bc15c None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:48.411335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0815e94-06c9-430f-9f6a-4e328c9bc15c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:48.411610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0815e94-06c9-430f-9f6a-4e328c9bc15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:48.442008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d0815e94-06c9-430f-9f6a-4e328c9bc15c None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:48.443264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d0815e94-06c9-430f-9f6a-4e328c9bc15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:48.519412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1269/2546] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:37:48.530306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9404a-3919-406a-9df0-19349a07ff22 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:48.530927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9404a-3919-406a-9df0-19349a07ff22 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:48.531232 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9404a-3919-406a-9df0-19349a07ff22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:48.557351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-92c9404a-3919-406a-9df0-19349a07ff22 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:48.558556 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-92c9404a-3919-406a-9df0-19349a07ff22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:48.566638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1278/2547] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:48.570987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0815e94-06c9-430f-9f6a-4e328c9bc15c None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:48.581385 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0815e94-06c9-430f-9f6a-4e328c9bc15c None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:48.586293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06cd6d-d1dd-4970-8363-812c9a0e2227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:48.586959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06cd6d-d1dd-4970-8363-812c9a0e2227 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:48.587233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06cd6d-d1dd-4970-8363-812c9a0e2227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:48.587883 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06cd6d-d1dd-4970-8363-812c9a0e2227 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:48.588192 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06cd6d-d1dd-4970-8363-812c9a0e2227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:37:48.588551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06cd6d-d1dd-4970-8363-812c9a0e2227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:48.590283 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06cd6d-d1dd-4970-8363-812c9a0e2227 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:48.605128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1270/2548] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:37:48.607441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9404a-3919-406a-9df0-19349a07ff22 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:48.630160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9404a-3919-406a-9df0-19349a07ff22 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:48.636204 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1688c-dfce-417d-a05f-db4896e87b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:48.636909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1688c-dfce-417d-a05f-db4896e87b1c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:48.637315 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1688c-dfce-417d-a05f-db4896e87b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:48.637683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1688c-dfce-417d-a05f-db4896e87b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:48.638365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1688c-dfce-417d-a05f-db4896e87b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:48.640560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1688c-dfce-417d-a05f-db4896e87b1c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:48.679848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1279/2549] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:37:48 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:37:48.680859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06cd6d-d1dd-4970-8363-812c9a0e2227 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:48.693916 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06cd6d-d1dd-4970-8363-812c9a0e2227 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:48.771241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e858f-4787-40c9-9be2-5b9a25758188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:48.772202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e858f-4787-40c9-9be2-5b9a25758188 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:48.772555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e858f-4787-40c9-9be2-5b9a25758188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:48.772896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e858f-4787-40c9-9be2-5b9a25758188 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:48.773702 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e858f-4787-40c9-9be2-5b9a25758188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:48.775762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e858f-4787-40c9-9be2-5b9a25758188 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:48.816456 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1271/2550] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:37:48 2022] POST /identity/v3/users => generated 286 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:37:48.820657 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1688c-dfce-417d-a05f-db4896e87b1c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:48.831436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1688c-dfce-417d-a05f-db4896e87b1c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:48.843874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a40854-14ca-485c-8a57-3c1272471aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:48.844658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a40854-14ca-485c-8a57-3c1272471aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:48.845028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a40854-14ca-485c-8a57-3c1272471aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:48.845491 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a40854-14ca-485c-8a57-3c1272471aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:48.846178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a40854-14ca-485c-8a57-3c1272471aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:48.848070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a40854-14ca-485c-8a57-3c1272471aa7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:48.862381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1280/2551] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:37:48.866359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e858f-4787-40c9-9be2-5b9a25758188 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:48.880669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e858f-4787-40c9-9be2-5b9a25758188 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:48.888039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938959b-05b8-47b6-978c-cd71b27f5ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:48.888665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938959b-05b8-47b6-978c-cd71b27f5ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:48.888939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938959b-05b8-47b6-978c-cd71b27f5ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:48.889531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938959b-05b8-47b6-978c-cd71b27f5ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:48.890210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938959b-05b8-47b6-978c-cd71b27f5ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:48.891827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938959b-05b8-47b6-978c-cd71b27f5ec3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:48.904676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1272/2552] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:37:48.910932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a40854-14ca-485c-8a57-3c1272471aa7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:48.921618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a40854-14ca-485c-8a57-3c1272471aa7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:48.926497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96258b0b-254b-4d00-8145-981acae2fd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:48.927162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96258b0b-254b-4d00-8145-981acae2fd30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:48.927437 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96258b0b-254b-4d00-8145-981acae2fd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:48.927717 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96258b0b-254b-4d00-8145-981acae2fd30 admin admin] PATH_INFO: `/v3/projects/db2188067a23494ab49f1e6812e501d6/users/7cd65e4d3b2f4976be9176e190d78d1d/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:48.947081 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96258b0b-254b-4d00-8145-981acae2fd30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db2188067a23494ab49f1e6812e501d6, user_id=7cd65e4d3b2f4976be9176e190d78d1d, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:48.950052 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96258b0b-254b-4d00-8145-981acae2fd30 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:48.965198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1281/2553] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:37:48 2022] PUT /identity/v3/projects/db2188067a23494ab49f1e6812e501d6/users/7cd65e4d3b2f4976be9176e190d78d1d/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:37:48.970909 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18199564-d380-4711-80e2-39aea27f29f4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:48.971182 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18199564-d380-4711-80e2-39aea27f29f4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:48.971519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18199564-d380-4711-80e2-39aea27f29f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:48.993787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-18199564-d380-4711-80e2-39aea27f29f4 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:48.995752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-18199564-d380-4711-80e2-39aea27f29f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:49.064513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1273/2554] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:49.074797 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96258b0b-254b-4d00-8145-981acae2fd30 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:49.076571 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-96258b0b-254b-4d00-8145-981acae2fd30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:49.141483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96258b0b-254b-4d00-8145-981acae2fd30 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:49.149101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56520a-7e38-4fa3-a1f4-2e33d3e829c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:49.149979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56520a-7e38-4fa3-a1f4-2e33d3e829c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:49.150319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56520a-7e38-4fa3-a1f4-2e33d3e829c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:49.150656 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56520a-7e38-4fa3-a1f4-2e33d3e829c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:49.161239 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab56520a-7e38-4fa3-a1f4-2e33d3e829c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:49.162804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab56520a-7e38-4fa3-a1f4-2e33d3e829c0 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:49.173900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1282/2555] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:49.919863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37d299-110a-4f78-91ed-833efd026a8e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:49.920448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37d299-110a-4f78-91ed-833efd026a8e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:49.920448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37d299-110a-4f78-91ed-833efd026a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:49.935337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:49.974109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6a37d299-110a-4f78-91ed-833efd026a8e None None] MFA Rules not processed for user `7cd65e4d3b2f4976be9176e190d78d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:49.975976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6a37d299-110a-4f78-91ed-833efd026a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:50.109918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1274/2556] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:37:50.120898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56520a-7e38-4fa3-a1f4-2e33d3e829c0 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:37:50.122901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ab56520a-7e38-4fa3-a1f4-2e33d3e829c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:50.176190 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:50.201899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56520a-7e38-4fa3-a1f4-2e33d3e829c0 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:37:50.210878 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533b8f4-689d-4577-888f-9790e1ca97fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:37:50.211688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533b8f4-689d-4577-888f-9790e1ca97fb service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:50.212219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533b8f4-689d-4577-888f-9790e1ca97fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:50.212857 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533b8f4-689d-4577-888f-9790e1ca97fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:50.228647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533b8f4-689d-4577-888f-9790e1ca97fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:37:50.231070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533b8f4-689d-4577-888f-9790e1ca97fb service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:37:50.246611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1283/2557] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:37:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:37:51.698738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbd054-b238-4db0-9a55-1ee819934305 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:37:51.699307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbd054-b238-4db0-9a55-1ee819934305 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:37:51.699307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbd054-b238-4db0-9a55-1ee819934305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:37:51.720033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-51fbd054-b238-4db0-9a55-1ee819934305 None None] MFA Rules not processed for user `7cd65e4d3b2f4976be9176e190d78d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:37:51.721199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-51fbd054-b238-4db0-9a55-1ee819934305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:37:51.732523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1275/2558] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:37:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:38:00.236931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=10 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:01.713469 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=33 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:03.639098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e2c15-d8c5-4736-99d9-3fe3a79c90be None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:03.639688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e2c15-d8c5-4736-99d9-3fe3a79c90be None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:03.639688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e2c15-d8c5-4736-99d9-3fe3a79c90be None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:03.641567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1284/2559] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:38:03.647675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d36db3-f51a-43d9-a33c-95f58d16edd9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:03.648168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d36db3-f51a-43d9-a33c-95f58d16edd9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:03.648269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d36db3-f51a-43d9-a33c-95f58d16edd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:03.672336 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-35d36db3-f51a-43d9-a33c-95f58d16edd9 None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:38:03.673620 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-35d36db3-f51a-43d9-a33c-95f58d16edd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:03.740017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1276/2560] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:38:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:38:03.752035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e2c15-d8c5-4736-99d9-3fe3a79c90be None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:38:03.753766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6e3e2c15-d8c5-4736-99d9-3fe3a79c90be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:03.809366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e2c15-d8c5-4736-99d9-3fe3a79c90be None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:38:03.812966 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d4ed8-d72e-4cb6-b20b-319e2e9adca3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf8a0ae03c94aa78cc2ab0e4e1c87b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:38:03.813616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d4ed8-d72e-4cb6-b20b-319e2e9adca3 service glance] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:03.813891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d4ed8-d72e-4cb6-b20b-319e2e9adca3 service glance] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:03.814165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d4ed8-d72e-4cb6-b20b-319e2e9adca3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:03.824062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d4ed8-d72e-4cb6-b20b-319e2e9adca3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:38:03.825541 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d4ed8-d72e-4cb6-b20b-319e2e9adca3 service glance] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:38:03.836006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1285/2561] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:38:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:38:03.910501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45357d8f-bcc5-400b-a7ad-bc250d14188d None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:03.910760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45357d8f-bcc5-400b-a7ad-bc250d14188d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:03.911048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45357d8f-bcc5-400b-a7ad-bc250d14188d None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:03.912401 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1277/2562] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 04:38:03.917196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46dce6f-0349-4da8-b67b-613402afee38 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:03.917478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46dce6f-0349-4da8-b67b-613402afee38 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:03.917750 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46dce6f-0349-4da8-b67b-613402afee38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:03.937776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a46dce6f-0349-4da8-b67b-613402afee38 None None] MFA Rules not processed for user `c9360be7ee9e4739b20c59955d28da47`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:38:03.939029 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a46dce6f-0349-4da8-b67b-613402afee38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:03.946035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1286/2563] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:38:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:38:03.960086 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45357d8f-bcc5-400b-a7ad-bc250d14188d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:38:03.962352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-45357d8f-bcc5-400b-a7ad-bc250d14188d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:03.978194 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45357d8f-bcc5-400b-a7ad-bc250d14188d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:38:03.982258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16646ce5-578c-43b1-88a2-3768448f9489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:38:03.982871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646ce5-578c-43b1-88a2-3768448f9489 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:03.983145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646ce5-578c-43b1-88a2-3768448f9489 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:03.983414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646ce5-578c-43b1-88a2-3768448f9489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:03.990852 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16646ce5-578c-43b1-88a2-3768448f9489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:38:03.992267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16646ce5-578c-43b1-88a2-3768448f9489 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:38:04.000172 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1278/2564] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:38:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:38:04.085533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0630c68-dc73-493d-8089-fcb51a610529 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:04.085819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0630c68-dc73-493d-8089-fcb51a610529 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:04.086098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0630c68-dc73-493d-8089-fcb51a610529 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:04.087469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1287/2565] 10.222.0.60 () {54 vars in 921 bytes} [Sat Sep 17 04:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 04:38:04.093410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45503f-1b91-41c5-b171-1cb18a09f581 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:04.093913 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45503f-1b91-41c5-b171-1cb18a09f581 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:04.094232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45503f-1b91-41c5-b171-1cb18a09f581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:04.114471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6c45503f-1b91-41c5-b171-1cb18a09f581 None None] MFA Rules not processed for user `d3272d57d5d24e5a990a6d7c86e6d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:38:04.115604 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6c45503f-1b91-41c5-b171-1cb18a09f581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:04.125278 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1279/2566] 10.222.0.60 () {58 vars in 1018 bytes} [Sat Sep 17 04:38:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:38:04.139207 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0630c68-dc73-493d-8089-fcb51a610529 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:38:04.150201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0630c68-dc73-493d-8089-fcb51a610529 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:38:04.153745 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbef6b1-6e84-43f2-831a-875907fcc170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:38:04.154332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbef6b1-6e84-43f2-831a-875907fcc170 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:04.154603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbef6b1-6e84-43f2-831a-875907fcc170 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:04.154862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbef6b1-6e84-43f2-831a-875907fcc170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:04.164526 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbef6b1-6e84-43f2-831a-875907fcc170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:38:04.165997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbef6b1-6e84-43f2-831a-875907fcc170 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:38:04.176275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1288/2567] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:38:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:38:07.160681 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608f33d-2360-4a39-b440-2ab47fbdbff1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:07.161349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608f33d-2360-4a39-b440-2ab47fbdbff1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:07.161349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608f33d-2360-4a39-b440-2ab47fbdbff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:07.181269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b608f33d-2360-4a39-b440-2ab47fbdbff1 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:38:07.182475 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b608f33d-2360-4a39-b440-2ab47fbdbff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:07.239444 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1280/2568] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:38:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:38:07.252128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbef6b1-6e84-43f2-831a-875907fcc170 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:38:07.253665 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4cbef6b1-6e84-43f2-831a-875907fcc170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:07.323645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbef6b1-6e84-43f2-831a-875907fcc170 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:38:07.328696 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b4280-dc9d-45d7-a46e-20ffe357c80c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:38:07.329348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b4280-dc9d-45d7-a46e-20ffe357c80c service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:07.329620 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b4280-dc9d-45d7-a46e-20ffe357c80c service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:07.330003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b4280-dc9d-45d7-a46e-20ffe357c80c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:07.338403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b4280-dc9d-45d7-a46e-20ffe357c80c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:38:07.340016 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b4280-dc9d-45d7-a46e-20ffe357c80c service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:38:07.348473 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1289/2569] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:38:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 04:38:11.773939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdbdcf-64e8-4b53-ace5-ac2946f88a3d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:11.774583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdbdcf-64e8-4b53-ace5-ac2946f88a3d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:11.774583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdbdcf-64e8-4b53-ace5-ac2946f88a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:11.794100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7bbdbdcf-64e8-4b53-ace5-ac2946f88a3d None None] MFA Rules not processed for user `ff241cbd3f7c47328ac5ef2ed4c66e18`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:38:11.795326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7bbdbdcf-64e8-4b53-ace5-ac2946f88a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:11.853159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1281/2570] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:38:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:38:17.345270 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=67 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:21.452074 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b4280-dc9d-45d7-a46e-20ffe357c80c service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:38:21.462093 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b4280-dc9d-45d7-a46e-20ffe357c80c service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:38:21.467646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce055a-c992-4b55-8a00-d68a6caeb36a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:38:21.468345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce055a-c992-4b55-8a00-d68a6caeb36a service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:21.468700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce055a-c992-4b55-8a00-d68a6caeb36a service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:21.469019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce055a-c992-4b55-8a00-d68a6caeb36a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:21.471338 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-19ce055a-c992-4b55-8a00-d68a6caeb36a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:21.485750 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce055a-c992-4b55-8a00-d68a6caeb36a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:38:21.487187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce055a-c992-4b55-8a00-d68a6caeb36a service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:38:21.498153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1290/2571] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:38:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:38:21.845619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:22.684426 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdbdcf-64e8-4b53-ace5-ac2946f88a3d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:38:22.694329 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdbdcf-64e8-4b53-ace5-ac2946f88a3d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:38:22.698107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b47bde-b1a6-4e6d-bcc0-0b4fa8a72827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:38:22.698697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47bde-b1a6-4e6d-bcc0-0b4fa8a72827 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:22.698971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47bde-b1a6-4e6d-bcc0-0b4fa8a72827 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:22.699239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47bde-b1a6-4e6d-bcc0-0b4fa8a72827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:22.701127 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-32b47bde-b1a6-4e6d-bcc0-0b4fa8a72827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:22.715225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b47bde-b1a6-4e6d-bcc0-0b4fa8a72827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:38:22.716746 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b47bde-b1a6-4e6d-bcc0-0b4fa8a72827 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:38:22.727801 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1282/2572] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:38:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:38:28.670866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42388ba4-37ba-4a8c-b1ab-7fa3a15d36a9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:28.671448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42388ba4-37ba-4a8c-b1ab-7fa3a15d36a9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:28.671448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42388ba4-37ba-4a8c-b1ab-7fa3a15d36a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:28.691198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-42388ba4-37ba-4a8c-b1ab-7fa3a15d36a9 None None] MFA Rules not processed for user `7cd65e4d3b2f4976be9176e190d78d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:38:28.692393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-42388ba4-37ba-4a8c-b1ab-7fa3a15d36a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:28.700633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1291/2573] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:38:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:38:30.010484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8431dd-a6c3-4cb5-bfec-fb26a218d8b3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:30.011032 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8431dd-a6c3-4cb5-bfec-fb26a218d8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:30.011032 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8431dd-a6c3-4cb5-bfec-fb26a218d8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:30.033296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ac8431dd-a6c3-4cb5-bfec-fb26a218d8b3 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:38:30.034635 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ac8431dd-a6c3-4cb5-bfec-fb26a218d8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:30.043193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1283/2574] 10.222.0.60 () {58 vars in 967 bytes} [Sat Sep 17 04:38:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:38:30.058902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42388ba4-37ba-4a8c-b1ab-7fa3a15d36a9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:38:30.072134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42388ba4-37ba-4a8c-b1ab-7fa3a15d36a9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:38:30.081873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2750298-0a9e-476f-886f-8d4ca85f8a8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:38:30.082748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2750298-0a9e-476f-886f-8d4ca85f8a8d service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:30.083273 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2750298-0a9e-476f-886f-8d4ca85f8a8d service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:30.083549 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2750298-0a9e-476f-886f-8d4ca85f8a8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:30.094398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2750298-0a9e-476f-886f-8d4ca85f8a8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:38:30.096214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2750298-0a9e-476f-886f-8d4ca85f8a8d service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:38:30.106563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1292/2575] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:38:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:38:30.233636 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8431dd-a6c3-4cb5-bfec-fb26a218d8b3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:38:30.246119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8431dd-a6c3-4cb5-bfec-fb26a218d8b3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:38:30.249908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34280cd5-9950-4efe-9a43-066ec5576638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:38:30.250550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34280cd5-9950-4efe-9a43-066ec5576638 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:30.250817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34280cd5-9950-4efe-9a43-066ec5576638 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:30.251298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34280cd5-9950-4efe-9a43-066ec5576638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:30.254313 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-34280cd5-9950-4efe-9a43-066ec5576638 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:30.364885 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34280cd5-9950-4efe-9a43-066ec5576638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:38:30.367096 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34280cd5-9950-4efe-9a43-066ec5576638 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:38:30.392471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1284/2576] 10.222.0.60 () {60 vars in 1396 bytes} [Sat Sep 17 04:38:30 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:38:40.102021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=15 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:40.389193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=50 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:47.836114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2750298-0a9e-476f-886f-8d4ca85f8a8d service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:38:47.846284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2750298-0a9e-476f-886f-8d4ca85f8a8d service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:38:47.849954 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe73f3-b914-4481-9b34-ef7106d6c735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:38:47.850555 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe73f3-b914-4481-9b34-ef7106d6c735 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:47.850851 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe73f3-b914-4481-9b34-ef7106d6c735 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:47.851134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe73f3-b914-4481-9b34-ef7106d6c735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:47.852969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-56fe73f3-b914-4481-9b34-ef7106d6c735 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:47.921246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe73f3-b914-4481-9b34-ef7106d6c735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:38:47.922693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe73f3-b914-4481-9b34-ef7106d6c735 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:38:47.933105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1293/2577] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:38:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:38:53.035769 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34280cd5-9950-4efe-9a43-066ec5576638 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:38:53.045075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34280cd5-9950-4efe-9a43-066ec5576638 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:38:53.049796 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a1fd8-20a2-44bf-8fba-d7942879e88d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:38:53.050407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a1fd8-20a2-44bf-8fba-d7942879e88d service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:53.050670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a1fd8-20a2-44bf-8fba-d7942879e88d service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:53.050967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a1fd8-20a2-44bf-8fba-d7942879e88d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:53.053010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-186a1fd8-20a2-44bf-8fba-d7942879e88d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:53.063320 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a1fd8-20a2-44bf-8fba-d7942879e88d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:38:53.064747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a1fd8-20a2-44bf-8fba-d7942879e88d service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:38:53.073560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1285/2578] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:38:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:38:55.017759 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de21aa-a99c-41ed-a0a7-b4ac6b0352b3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:55.018470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de21aa-a99c-41ed-a0a7-b4ac6b0352b3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:55.018470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de21aa-a99c-41ed-a0a7-b4ac6b0352b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:55.069764 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f7de21aa-a99c-41ed-a0a7-b4ac6b0352b3 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:38:55.070980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f7de21aa-a99c-41ed-a0a7-b4ac6b0352b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:38:55.087776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1294/2579] 10.222.0.60 () {58 vars in 1009 bytes} [Sat Sep 17 04:38:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:38:55.109781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a1fd8-20a2-44bf-8fba-d7942879e88d service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:38:55.132506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a1fd8-20a2-44bf-8fba-d7942879e88d service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:38:55.138186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347954c8-71fb-4e39-888d-c0cc17f31b04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:38:55.138802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347954c8-71fb-4e39-888d-c0cc17f31b04 service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:38:55.139060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347954c8-71fb-4e39-888d-c0cc17f31b04 service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:38:55.139328 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347954c8-71fb-4e39-888d-c0cc17f31b04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:38:55.147708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347954c8-71fb-4e39-888d-c0cc17f31b04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:38:55.149359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347954c8-71fb-4e39-888d-c0cc17f31b04 service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:38:55.157445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1286/2580] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:38:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:39:05.084830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=48 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:05.154212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=16 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:10.566969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080001a9-7a32-45ca-ad1b-947583df2b02 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:10.567410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080001a9-7a32-45ca-ad1b-947583df2b02 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:10.567503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080001a9-7a32-45ca-ad1b-947583df2b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:10.587580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-080001a9-7a32-45ca-ad1b-947583df2b02 None None] MFA Rules not processed for user `7c1a7d2677184c24ade81c397b7d09d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:39:10.588791 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-080001a9-7a32-45ca-ad1b-947583df2b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:10.597802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1295/2581] 10.222.0.60 () {58 vars in 967 bytes} [Sat Sep 17 04:39:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:39:10.610249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347954c8-71fb-4e39-888d-c0cc17f31b04 service swift] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:10.621958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347954c8-71fb-4e39-888d-c0cc17f31b04 service swift] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:10.628968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677992b3-0a13-4e9e-8aa2-45af9675e64e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aac7cd1742449328facf2902483f259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:10.629610 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677992b3-0a13-4e9e-8aa2-45af9675e64e service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:10.629894 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677992b3-0a13-4e9e-8aa2-45af9675e64e service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:10.630161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677992b3-0a13-4e9e-8aa2-45af9675e64e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:10.639694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677992b3-0a13-4e9e-8aa2-45af9675e64e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:10.641331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677992b3-0a13-4e9e-8aa2-45af9675e64e service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:10.649524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1287/2582] 10.222.0.60 () {60 vars in 1384 bytes} [Sat Sep 17 04:39:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 04:39:12.547835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080001a9-7a32-45ca-ad1b-947583df2b02 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:12.549420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-080001a9-7a32-45ca-ad1b-947583df2b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:12.625278 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080001a9-7a32-45ca-ad1b-947583df2b02 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:12.630396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd385686-f0ce-4a6d-9a3d-98451e377165 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:12.631023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd385686-f0ce-4a6d-9a3d-98451e377165 service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:12.631307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd385686-f0ce-4a6d-9a3d-98451e377165 service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:12.631579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd385686-f0ce-4a6d-9a3d-98451e377165 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:12.633447 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cd385686-f0ce-4a6d-9a3d-98451e377165 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:12.646054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd385686-f0ce-4a6d-9a3d-98451e377165 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:12.647695 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd385686-f0ce-4a6d-9a3d-98451e377165 service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:12.661226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1296/2583] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:39:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:39:12.890426 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677992b3-0a13-4e9e-8aa2-45af9675e64e service swift] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:12.901827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677992b3-0a13-4e9e-8aa2-45af9675e64e service swift] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:12.907387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b41ab-1103-4a84-b2f9-a18846127114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:12.908026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b41ab-1103-4a84-b2f9-a18846127114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:12.908305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b41ab-1103-4a84-b2f9-a18846127114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:12.908567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b41ab-1103-4a84-b2f9-a18846127114 admin admin] PATH_INFO: `/v3/users/7cd65e4d3b2f4976be9176e190d78d1d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:12.909742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b41ab-1103-4a84-b2f9-a18846127114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cd65e4d3b2f4976be9176e190d78d1d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:12.911660 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b41ab-1103-4a84-b2f9-a18846127114 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:12.960466 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d75b41ab-1103-4a84-b2f9-a18846127114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cd65e4d3b2f4976be9176e190d78d1d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:12.969937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d75b41ab-1103-4a84-b2f9-a18846127114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cd65e4d3b2f4976be9176e190d78d1d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:12.970600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d75b41ab-1103-4a84-b2f9-a18846127114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cd65e4d3b2f4976be9176e190d78d1d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:12.977819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d75b41ab-1103-4a84-b2f9-a18846127114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cd65e4d3b2f4976be9176e190d78d1d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:12.989676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d75b41ab-1103-4a84-b2f9-a18846127114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cd65e4d3b2f4976be9176e190d78d1d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:12.995493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1288/2584] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:39:12 2022] DELETE /identity/v3/users/7cd65e4d3b2f4976be9176e190d78d1d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:39:13.201834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd385686-f0ce-4a6d-9a3d-98451e377165 service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:13.213669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd385686-f0ce-4a6d-9a3d-98451e377165 service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:13.277250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde7649-5602-4009-8805-6f4b3a6b7ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:13.277899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde7649-5602-4009-8805-6f4b3a6b7ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:13.278158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde7649-5602-4009-8805-6f4b3a6b7ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:13.278422 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde7649-5602-4009-8805-6f4b3a6b7ee7 admin admin] PATH_INFO: `/v3/projects/db2188067a23494ab49f1e6812e501d6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:13.279564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde7649-5602-4009-8805-6f4b3a6b7ee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db2188067a23494ab49f1e6812e501d6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:13.281515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde7649-5602-4009-8805-6f4b3a6b7ee7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:13.328010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bde7649-5602-4009-8805-6f4b3a6b7ee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db2188067a23494ab49f1e6812e501d6', 'request_id': 'req-3bde7649-5602-4009-8805-6f4b3a6b7ee7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:13.335807 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bde7649-5602-4009-8805-6f4b3a6b7ee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db2188067a23494ab49f1e6812e501d6', 'request_id': 'req-3bde7649-5602-4009-8805-6f4b3a6b7ee7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:13.342049 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bde7649-5602-4009-8805-6f4b3a6b7ee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db2188067a23494ab49f1e6812e501d6', 'request_id': 'req-3bde7649-5602-4009-8805-6f4b3a6b7ee7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:13.352982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bde7649-5602-4009-8805-6f4b3a6b7ee7 admin admin] The token cache is being invalidate because project db2188067a23494ab49f1e6812e501d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:39:13.353303 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bde7649-5602-4009-8805-6f4b3a6b7ee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:13.354015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bde7649-5602-4009-8805-6f4b3a6b7ee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:13.355549 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1297/2585] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:39:13 2022] DELETE /identity/v3/projects/db2188067a23494ab49f1e6812e501d6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:39:13.361228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefbf29-4e48-44c5-8781-74ac7ffad37e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:13.361561 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefbf29-4e48-44c5-8781-74ac7ffad37e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:13.361835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefbf29-4e48-44c5-8781-74ac7ffad37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:13.381180 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1cefbf29-4e48-44c5-8781-74ac7ffad37e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:39:13.382372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1cefbf29-4e48-44c5-8781-74ac7ffad37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:13.449459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1289/2586] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:39:13.459009 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112a96a-7ff3-4c65-bc27-b51959e62d95 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:13.459311 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112a96a-7ff3-4c65-bc27-b51959e62d95 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:13.459585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112a96a-7ff3-4c65-bc27-b51959e62d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:13.482725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5112a96a-7ff3-4c65-bc27-b51959e62d95 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:39:13.483996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5112a96a-7ff3-4c65-bc27-b51959e62d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:13.492739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1298/2587] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:39:13.497043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefbf29-4e48-44c5-8781-74ac7ffad37e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:13.506697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefbf29-4e48-44c5-8781-74ac7ffad37e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:13.511272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40785ae1-1790-467d-81f4-6ba04b1d2128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:13.511867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40785ae1-1790-467d-81f4-6ba04b1d2128 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:13.512135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40785ae1-1790-467d-81f4-6ba04b1d2128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:13.512401 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40785ae1-1790-467d-81f4-6ba04b1d2128 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:13.513039 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40785ae1-1790-467d-81f4-6ba04b1d2128 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:39:13.513374 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40785ae1-1790-467d-81f4-6ba04b1d2128 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:13.514951 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40785ae1-1790-467d-81f4-6ba04b1d2128 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:13.526521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1290/2588] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:39:13.530855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112a96a-7ff3-4c65-bc27-b51959e62d95 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:13.539987 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112a96a-7ff3-4c65-bc27-b51959e62d95 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:13.544924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71c19c-34c3-4cfe-a96a-5d8804a0e8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:13.545548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71c19c-34c3-4cfe-a96a-5d8804a0e8cc admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:13.545822 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71c19c-34c3-4cfe-a96a-5d8804a0e8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:13.546085 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71c19c-34c3-4cfe-a96a-5d8804a0e8cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:13.546877 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71c19c-34c3-4cfe-a96a-5d8804a0e8cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:13.548780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71c19c-34c3-4cfe-a96a-5d8804a0e8cc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:13.553011 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:13.597743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1299/2589] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:39:13 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:39:13.602023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40785ae1-1790-467d-81f4-6ba04b1d2128 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:13.611928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40785ae1-1790-467d-81f4-6ba04b1d2128 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:13.659776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:13.678794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a3bb4-561a-4b2e-a06e-2e8a114243c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:13.679415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a3bb4-561a-4b2e-a06e-2e8a114243c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:13.679689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a3bb4-561a-4b2e-a06e-2e8a114243c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:13.680096 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a3bb4-561a-4b2e-a06e-2e8a114243c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:13.680730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a3bb4-561a-4b2e-a06e-2e8a114243c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:13.682669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a3bb4-561a-4b2e-a06e-2e8a114243c7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:13.708045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1291/2590] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:39:13 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:39:13.712262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71c19c-34c3-4cfe-a96a-5d8804a0e8cc admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:13.723083 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71c19c-34c3-4cfe-a96a-5d8804a0e8cc admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:13.727873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d50b6-00c1-49b5-9ed2-7fece9e73bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:13.728473 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d50b6-00c1-49b5-9ed2-7fece9e73bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:13.728749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d50b6-00c1-49b5-9ed2-7fece9e73bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:13.729018 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d50b6-00c1-49b5-9ed2-7fece9e73bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:13.729680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d50b6-00c1-49b5-9ed2-7fece9e73bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:13.731321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d50b6-00c1-49b5-9ed2-7fece9e73bf0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:13.741413 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1300/2591] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:39:13.745533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a3bb4-561a-4b2e-a06e-2e8a114243c7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:13.755782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a3bb4-561a-4b2e-a06e-2e8a114243c7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:13.760901 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9b2ed-6bfd-4da0-98b0-5c0186aa336a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:13.761507 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9b2ed-6bfd-4da0-98b0-5c0186aa336a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:13.761819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9b2ed-6bfd-4da0-98b0-5c0186aa336a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:13.762091 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9b2ed-6bfd-4da0-98b0-5c0186aa336a admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:13.762705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9b2ed-6bfd-4da0-98b0-5c0186aa336a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:13.764350 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9b2ed-6bfd-4da0-98b0-5c0186aa336a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:13.775201 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1292/2592] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:39:13.779538 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d50b6-00c1-49b5-9ed2-7fece9e73bf0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:13.789517 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d50b6-00c1-49b5-9ed2-7fece9e73bf0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:13.794548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7efee88-0e10-43f4-9213-5374036c9f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:13.795199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efee88-0e10-43f4-9213-5374036c9f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:13.795474 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efee88-0e10-43f4-9213-5374036c9f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:13.795744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efee88-0e10-43f4-9213-5374036c9f6c admin admin] PATH_INFO: `/v3/projects/d65fd46cdb484e72a74fad241e81130b/users/65baeaf474294b24b5e7e7406932078e/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:13.811123 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7efee88-0e10-43f4-9213-5374036c9f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65fd46cdb484e72a74fad241e81130b, user_id=65baeaf474294b24b5e7e7406932078e, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:13.813888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7efee88-0e10-43f4-9213-5374036c9f6c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:13.829608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1301/2593] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:39:13 2022] PUT /identity/v3/projects/d65fd46cdb484e72a74fad241e81130b/users/65baeaf474294b24b5e7e7406932078e/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:39:13.841373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e520b41-1f88-4aed-9dbe-28b5300632e9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:13.841657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e520b41-1f88-4aed-9dbe-28b5300632e9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:13.841925 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e520b41-1f88-4aed-9dbe-28b5300632e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:13.871856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7e520b41-1f88-4aed-9dbe-28b5300632e9 None None] MFA Rules not processed for user `65baeaf474294b24b5e7e7406932078e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:39:13.873059 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7e520b41-1f88-4aed-9dbe-28b5300632e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:13.939190 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1293/2594] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:39:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:39:13.951091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7efee88-0e10-43f4-9213-5374036c9f6c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:13.953116 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f7efee88-0e10-43f4-9213-5374036c9f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:14.013917 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7efee88-0e10-43f4-9213-5374036c9f6c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:14.018012 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f143a-3126-42c8-889e-260f9cac9bb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:14.018613 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f143a-3126-42c8-889e-260f9cac9bb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:14.018890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f143a-3126-42c8-889e-260f9cac9bb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:14.019162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f143a-3126-42c8-889e-260f9cac9bb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:14.027667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f143a-3126-42c8-889e-260f9cac9bb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:14.029131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f143a-3126-42c8-889e-260f9cac9bb4 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:14.037939 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1302/2595] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:39:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:39:16.638190 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e520b41-1f88-4aed-9dbe-28b5300632e9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:16.706166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e520b41-1f88-4aed-9dbe-28b5300632e9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:16.711496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:16.712134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:16.712411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:16.712676 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] PATH_INFO: `/v3/users/65baeaf474294b24b5e7e7406932078e` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:16.714284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=65baeaf474294b24b5e7e7406932078e)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:16.716259 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:16.763880 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65baeaf474294b24b5e7e7406932078e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:16.770951 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65baeaf474294b24b5e7e7406932078e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:16.771632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65baeaf474294b24b5e7e7406932078e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:16.773626 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:16.779400 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65baeaf474294b24b5e7e7406932078e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:16.792214 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65baeaf474294b24b5e7e7406932078e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:16.797901 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1294/2596] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:39:16 2022] DELETE /identity/v3/users/65baeaf474294b24b5e7e7406932078e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:39:16.802566 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f6e8c-5b47-4a45-8dee-65dfd6e87920 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:16.803038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f6e8c-5b47-4a45-8dee-65dfd6e87920 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:16.803172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f6e8c-5b47-4a45-8dee-65dfd6e87920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:16.822262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c08f6e8c-5b47-4a45-8dee-65dfd6e87920 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:39:16.823457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c08f6e8c-5b47-4a45-8dee-65dfd6e87920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:16.886941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1303/2597] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:39:16.898432 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:16.900461 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:16.959416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf26b6a-d7c3-4a50-a567-cd5af83c7bcb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:16.964203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44c98-be7f-4be5-905c-5b1e28166026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:16.964817 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44c98-be7f-4be5-905c-5b1e28166026 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:16.965124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44c98-be7f-4be5-905c-5b1e28166026 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:16.965389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44c98-be7f-4be5-905c-5b1e28166026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:16.975576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b44c98-be7f-4be5-905c-5b1e28166026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:16.977042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b44c98-be7f-4be5-905c-5b1e28166026 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:16.987144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1295/2598] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:39:17.275212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f6e8c-5b47-4a45-8dee-65dfd6e87920 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:17.276702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c08f6e8c-5b47-4a45-8dee-65dfd6e87920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:17.288956 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f6e8c-5b47-4a45-8dee-65dfd6e87920 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:17.293751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:17.294393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:17.294668 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:17.294940 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] PATH_INFO: `/v3/projects/d65fd46cdb484e72a74fad241e81130b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:17.296576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d65fd46cdb484e72a74fad241e81130b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:17.298527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:17.341712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:17.349880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd65fd46cdb484e72a74fad241e81130b', 'request_id': 'req-4fa2a898-af14-44ba-834b-384eb1be38cf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:17.358171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd65fd46cdb484e72a74fad241e81130b', 'request_id': 'req-4fa2a898-af14-44ba-834b-384eb1be38cf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:17.363779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd65fd46cdb484e72a74fad241e81130b', 'request_id': 'req-4fa2a898-af14-44ba-834b-384eb1be38cf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:17.374937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] The token cache is being invalidate because project d65fd46cdb484e72a74fad241e81130b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:39:17.375227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:17.375940 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:17.377564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1304/2599] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:39:17 2022] DELETE /identity/v3/projects/d65fd46cdb484e72a74fad241e81130b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:39:26.982667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=36,keystone:DELETE=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:27.371744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:44.615752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44c98-be7f-4be5-905c-5b1e28166026 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:44.624609 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44c98-be7f-4be5-905c-5b1e28166026 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:44.678100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90625c-6ffe-44b1-8c54-39d7bdf3bbeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:44.678720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90625c-6ffe-44b1-8c54-39d7bdf3bbeb service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:44.679001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90625c-6ffe-44b1-8c54-39d7bdf3bbeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:44.679274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90625c-6ffe-44b1-8c54-39d7bdf3bbeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:44.681186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5b90625c-6ffe-44b1-8c54-39d7bdf3bbeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:44.747582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90625c-6ffe-44b1-8c54-39d7bdf3bbeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:44.748938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90625c-6ffe-44b1-8c54-39d7bdf3bbeb service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:44.759040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1296/2600] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:39:49.198359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:49.201795 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:49.226468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2a898-af14-44ba-834b-384eb1be38cf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:49.232023 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa39f65-15ca-4d1e-9be9-1c563ba04892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:49.232672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa39f65-15ca-4d1e-9be9-1c563ba04892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:49.233164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa39f65-15ca-4d1e-9be9-1c563ba04892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:49.233269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa39f65-15ca-4d1e-9be9-1c563ba04892 admin admin] PATH_INFO: `/v3/users/094abcdcd1f54a658564357056e7849a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:49.248595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa39f65-15ca-4d1e-9be9-1c563ba04892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=094abcdcd1f54a658564357056e7849a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:49.250882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa39f65-15ca-4d1e-9be9-1c563ba04892 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:49.300106 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9fa39f65-15ca-4d1e-9be9-1c563ba04892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '094abcdcd1f54a658564357056e7849a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:49.308973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9fa39f65-15ca-4d1e-9be9-1c563ba04892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '094abcdcd1f54a658564357056e7849a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:49.309626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9fa39f65-15ca-4d1e-9be9-1c563ba04892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '094abcdcd1f54a658564357056e7849a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:49.316724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9fa39f65-15ca-4d1e-9be9-1c563ba04892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '094abcdcd1f54a658564357056e7849a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:49.329399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9fa39f65-15ca-4d1e-9be9-1c563ba04892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '094abcdcd1f54a658564357056e7849a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:49.335619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1305/2601] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:39:49 2022] DELETE /identity/v3/users/094abcdcd1f54a658564357056e7849a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:39:49.511600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90625c-6ffe-44b1-8c54-39d7bdf3bbeb service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:49.522814 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90625c-6ffe-44b1-8c54-39d7bdf3bbeb service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:49.582414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a6466-625b-4709-ae6a-600d809b4045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:49.583042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a6466-625b-4709-ae6a-600d809b4045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:49.583312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a6466-625b-4709-ae6a-600d809b4045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:49.583607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a6466-625b-4709-ae6a-600d809b4045 admin admin] PATH_INFO: `/v3/projects/e9f593f419bb4ebeb351a9ad54a2a532` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:49.594701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a6466-625b-4709-ae6a-600d809b4045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f593f419bb4ebeb351a9ad54a2a532)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:49.596682 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a6466-625b-4709-ae6a-600d809b4045 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:49.642962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:49.647236 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-457a6466-625b-4709-ae6a-600d809b4045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f593f419bb4ebeb351a9ad54a2a532', 'request_id': 'req-457a6466-625b-4709-ae6a-600d809b4045'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:49.655762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-457a6466-625b-4709-ae6a-600d809b4045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f593f419bb4ebeb351a9ad54a2a532', 'request_id': 'req-457a6466-625b-4709-ae6a-600d809b4045'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:49.662195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-457a6466-625b-4709-ae6a-600d809b4045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f593f419bb4ebeb351a9ad54a2a532', 'request_id': 'req-457a6466-625b-4709-ae6a-600d809b4045'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:49.680230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-457a6466-625b-4709-ae6a-600d809b4045 admin admin] The token cache is being invalidate because project e9f593f419bb4ebeb351a9ad54a2a532 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:39:49.680524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-457a6466-625b-4709-ae6a-600d809b4045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:49.681300 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-457a6466-625b-4709-ae6a-600d809b4045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:49.682871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1297/2602] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:39:49 2022] DELETE /identity/v3/projects/e9f593f419bb4ebeb351a9ad54a2a532 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:39:49.688100 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5589da-981a-4af5-8b95-2bc631ff5c89 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:49.688384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5589da-981a-4af5-8b95-2bc631ff5c89 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:49.688651 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5589da-981a-4af5-8b95-2bc631ff5c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:49.710536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1b5589da-981a-4af5-8b95-2bc631ff5c89 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:39:49.711763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1b5589da-981a-4af5-8b95-2bc631ff5c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:49.775604 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1306/2603] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:39:49.783771 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b0bf4-6564-4652-938a-4767b3c784e7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:49.784056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b0bf4-6564-4652-938a-4767b3c784e7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:49.784340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b0bf4-6564-4652-938a-4767b3c784e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:49.929251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-021b0bf4-6564-4652-938a-4767b3c784e7 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:39:49.930409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-021b0bf4-6564-4652-938a-4767b3c784e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:49.939818 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1298/2604] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:39:49.944686 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5589da-981a-4af5-8b95-2bc631ff5c89 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:49.955553 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5589da-981a-4af5-8b95-2bc631ff5c89 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:49.961121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fd19c-612a-4d6f-b9b0-4a5d76bb21c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:49.961735 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fd19c-612a-4d6f-b9b0-4a5d76bb21c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:49.961986 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fd19c-612a-4d6f-b9b0-4a5d76bb21c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:49.962302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fd19c-612a-4d6f-b9b0-4a5d76bb21c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:49.962950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fd19c-612a-4d6f-b9b0-4a5d76bb21c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:39:49.963240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fd19c-612a-4d6f-b9b0-4a5d76bb21c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:49.965216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fd19c-612a-4d6f-b9b0-4a5d76bb21c9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:49.977405 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1307/2605] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:39:49.982124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b0bf4-6564-4652-938a-4767b3c784e7 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:49.991788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b0bf4-6564-4652-938a-4767b3c784e7 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:49.997669 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d273-4122-4ac1-b05a-59e49705088f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:49.998272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d273-4122-4ac1-b05a-59e49705088f admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:49.998547 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d273-4122-4ac1-b05a-59e49705088f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:49.998815 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d273-4122-4ac1-b05a-59e49705088f admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:49.999440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7d273-4122-4ac1-b05a-59e49705088f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:50.001346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7d273-4122-4ac1-b05a-59e49705088f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:50.031377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1299/2606] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:39:49 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:39:50.035861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fd19c-612a-4d6f-b9b0-4a5d76bb21c9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:50.046254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fd19c-612a-4d6f-b9b0-4a5d76bb21c9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:50.092638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:50.107154 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e2fc7-acb7-4355-b57d-f2f7c971bdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:50.107795 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e2fc7-acb7-4355-b57d-f2f7c971bdd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:50.108073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e2fc7-acb7-4355-b57d-f2f7c971bdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:50.108344 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e2fc7-acb7-4355-b57d-f2f7c971bdd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:50.108998 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e2fc7-acb7-4355-b57d-f2f7c971bdd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:50.111217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e2fc7-acb7-4355-b57d-f2f7c971bdd8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:50.133901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1308/2607] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:39:50 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:39:50.138427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d273-4122-4ac1-b05a-59e49705088f admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:50.149423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d273-4122-4ac1-b05a-59e49705088f admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:50.154369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8a363-7ebb-49b5-ac6a-c29ffbe25511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:50.154965 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8a363-7ebb-49b5-ac6a-c29ffbe25511 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:50.155241 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8a363-7ebb-49b5-ac6a-c29ffbe25511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:50.155510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8a363-7ebb-49b5-ac6a-c29ffbe25511 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:50.156168 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8a363-7ebb-49b5-ac6a-c29ffbe25511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:50.157782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8a363-7ebb-49b5-ac6a-c29ffbe25511 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:50.167716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1300/2608] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:39:50.172180 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e2fc7-acb7-4355-b57d-f2f7c971bdd8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:50.184408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e2fc7-acb7-4355-b57d-f2f7c971bdd8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:50.190067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fb4f8-7889-4434-840c-8a324cdbde22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:50.190669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fb4f8-7889-4434-840c-8a324cdbde22 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:50.190951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fb4f8-7889-4434-840c-8a324cdbde22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:50.191220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fb4f8-7889-4434-840c-8a324cdbde22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:50.191860 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fb4f8-7889-4434-840c-8a324cdbde22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:50.193649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fb4f8-7889-4434-840c-8a324cdbde22 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:50.205006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1309/2609] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:39:50.211297 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8a363-7ebb-49b5-ac6a-c29ffbe25511 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:50.225519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8a363-7ebb-49b5-ac6a-c29ffbe25511 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:50.230650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42217a4f-68e7-4a50-ada9-ef9eb15a3248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:50.231375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42217a4f-68e7-4a50-ada9-ef9eb15a3248 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:50.231651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42217a4f-68e7-4a50-ada9-ef9eb15a3248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:50.231921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42217a4f-68e7-4a50-ada9-ef9eb15a3248 admin admin] PATH_INFO: `/v3/projects/f5eef26df9ed4b079274385534f4ec40/users/70bb0b4438f845a998daeafb85504dec/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:50.247612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42217a4f-68e7-4a50-ada9-ef9eb15a3248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5eef26df9ed4b079274385534f4ec40, user_id=70bb0b4438f845a998daeafb85504dec, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:50.250283 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42217a4f-68e7-4a50-ada9-ef9eb15a3248 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:50.264781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1301/2610] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:39:50 2022] PUT /identity/v3/projects/f5eef26df9ed4b079274385534f4ec40/users/70bb0b4438f845a998daeafb85504dec/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:39:50.275202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a2cd6-d6b7-4faa-a984-0f4bae4e53ff None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:50.275486 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a2cd6-d6b7-4faa-a984-0f4bae4e53ff None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:50.275758 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a2cd6-d6b7-4faa-a984-0f4bae4e53ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:50.304948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-718a2cd6-d6b7-4faa-a984-0f4bae4e53ff None None] MFA Rules not processed for user `70bb0b4438f845a998daeafb85504dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:39:50.306251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-718a2cd6-d6b7-4faa-a984-0f4bae4e53ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:50.374480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1310/2611] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:39:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:39:50.385525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42217a4f-68e7-4a50-ada9-ef9eb15a3248 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:50.391171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-42217a4f-68e7-4a50-ada9-ef9eb15a3248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:50.471041 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42217a4f-68e7-4a50-ada9-ef9eb15a3248 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:50.476827 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12ea4f-8c6a-462e-894d-61b6258a806d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:50.477456 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12ea4f-8c6a-462e-894d-61b6258a806d service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:50.477765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12ea4f-8c6a-462e-894d-61b6258a806d service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:50.478003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12ea4f-8c6a-462e-894d-61b6258a806d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:50.489662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12ea4f-8c6a-462e-894d-61b6258a806d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:50.492922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12ea4f-8c6a-462e-894d-61b6258a806d service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:50.502434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1302/2612] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:39:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:39:51.653403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a2cd6-d6b7-4faa-a984-0f4bae4e53ff None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:51.654908 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-718a2cd6-d6b7-4faa-a984-0f4bae4e53ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:51.716925 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a2cd6-d6b7-4faa-a984-0f4bae4e53ff None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:51.721554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff0443-f973-4681-991a-2c6c425a5ebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:51.722165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff0443-f973-4681-991a-2c6c425a5ebb service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:51.722774 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff0443-f973-4681-991a-2c6c425a5ebb service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:51.723054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff0443-f973-4681-991a-2c6c425a5ebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:51.725488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ecff0443-f973-4681-991a-2c6c425a5ebb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:51.794192 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff0443-f973-4681-991a-2c6c425a5ebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:51.795633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff0443-f973-4681-991a-2c6c425a5ebb service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:51.805716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1311/2613] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:39:55.032701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12ea4f-8c6a-462e-894d-61b6258a806d service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:55.034220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5e12ea4f-8c6a-462e-894d-61b6258a806d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:55.049591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12ea4f-8c6a-462e-894d-61b6258a806d service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:55.054674 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30082bdf-56fd-416e-b619-2c5ca00ad1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:55.055314 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30082bdf-56fd-416e-b619-2c5ca00ad1cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:55.055590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30082bdf-56fd-416e-b619-2c5ca00ad1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:55.055889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30082bdf-56fd-416e-b619-2c5ca00ad1cd admin admin] PATH_INFO: `/v3/users/ed4e695113f843fda52ce62792a36bb0` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:55.057006 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30082bdf-56fd-416e-b619-2c5ca00ad1cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4e695113f843fda52ce62792a36bb0)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:55.059015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30082bdf-56fd-416e-b619-2c5ca00ad1cd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:55.116128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30082bdf-56fd-416e-b619-2c5ca00ad1cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4e695113f843fda52ce62792a36bb0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:55.118046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:55.125196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30082bdf-56fd-416e-b619-2c5ca00ad1cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4e695113f843fda52ce62792a36bb0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:55.126288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30082bdf-56fd-416e-b619-2c5ca00ad1cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4e695113f843fda52ce62792a36bb0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:55.135895 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30082bdf-56fd-416e-b619-2c5ca00ad1cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4e695113f843fda52ce62792a36bb0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:55.151884 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30082bdf-56fd-416e-b619-2c5ca00ad1cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4e695113f843fda52ce62792a36bb0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:55.161505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1303/2614] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:39:55 2022] DELETE /identity/v3/users/ed4e695113f843fda52ce62792a36bb0 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:39:55.355261 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff0443-f973-4681-991a-2c6c425a5ebb service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:55.369216 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff0443-f973-4681-991a-2c6c425a5ebb service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:55.373636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:55.431159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcadb3b-98de-4fab-85be-1844fa072b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:55.431797 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcadb3b-98de-4fab-85be-1844fa072b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:55.432080 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcadb3b-98de-4fab-85be-1844fa072b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:55.432352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcadb3b-98de-4fab-85be-1844fa072b88 admin admin] PATH_INFO: `/v3/projects/e0b194b28dec41119ae0d6d913f19c31` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:55.433494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcadb3b-98de-4fab-85be-1844fa072b88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b194b28dec41119ae0d6d913f19c31)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:55.435417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcadb3b-98de-4fab-85be-1844fa072b88 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:55.494138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5fcadb3b-98de-4fab-85be-1844fa072b88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b194b28dec41119ae0d6d913f19c31', 'request_id': 'req-5fcadb3b-98de-4fab-85be-1844fa072b88'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:55.502726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5fcadb3b-98de-4fab-85be-1844fa072b88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b194b28dec41119ae0d6d913f19c31', 'request_id': 'req-5fcadb3b-98de-4fab-85be-1844fa072b88'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:55.508780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5fcadb3b-98de-4fab-85be-1844fa072b88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b194b28dec41119ae0d6d913f19c31', 'request_id': 'req-5fcadb3b-98de-4fab-85be-1844fa072b88'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:55.519974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5fcadb3b-98de-4fab-85be-1844fa072b88 admin admin] The token cache is being invalidate because project e0b194b28dec41119ae0d6d913f19c31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:39:55.520256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5fcadb3b-98de-4fab-85be-1844fa072b88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:55.520894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5fcadb3b-98de-4fab-85be-1844fa072b88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:39:55.522424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1312/2615] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:39:55 2022] DELETE /identity/v3/projects/e0b194b28dec41119ae0d6d913f19c31 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:39:55.527734 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24318c6a-ce0a-4001-9bf7-5cf4afd4d298 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:55.528023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24318c6a-ce0a-4001-9bf7-5cf4afd4d298 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:55.528289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24318c6a-ce0a-4001-9bf7-5cf4afd4d298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:55.548362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-24318c6a-ce0a-4001-9bf7-5cf4afd4d298 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:39:55.549606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-24318c6a-ce0a-4001-9bf7-5cf4afd4d298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:55.611604 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1304/2616] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:39:55.620368 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295ec94-efb0-4472-a20a-f25fcf1ef0af None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:55.620703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295ec94-efb0-4472-a20a-f25fcf1ef0af None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:55.620997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295ec94-efb0-4472-a20a-f25fcf1ef0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:55.771217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f295ec94-efb0-4472-a20a-f25fcf1ef0af None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:39:55.772507 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f295ec94-efb0-4472-a20a-f25fcf1ef0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:55.781138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1313/2617] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:39:55.785675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24318c6a-ce0a-4001-9bf7-5cf4afd4d298 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:55.795687 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24318c6a-ce0a-4001-9bf7-5cf4afd4d298 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:55.800874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac582b-eaae-4894-a1e3-20e79eb3f11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:55.801506 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac582b-eaae-4894-a1e3-20e79eb3f11a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:55.801782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac582b-eaae-4894-a1e3-20e79eb3f11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:55.802047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac582b-eaae-4894-a1e3-20e79eb3f11a admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:55.802764 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac582b-eaae-4894-a1e3-20e79eb3f11a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:39:55.803061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac582b-eaae-4894-a1e3-20e79eb3f11a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:55.804632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac582b-eaae-4894-a1e3-20e79eb3f11a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:55.821009 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1305/2618] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:39:55.826198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295ec94-efb0-4472-a20a-f25fcf1ef0af None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:55.838065 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295ec94-efb0-4472-a20a-f25fcf1ef0af None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:55.843348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c35a1-86ff-4ef3-9463-a2c0c6311c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:55.843945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c35a1-86ff-4ef3-9463-a2c0c6311c5c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:55.844221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c35a1-86ff-4ef3-9463-a2c0c6311c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:55.844489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c35a1-86ff-4ef3-9463-a2c0c6311c5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:55.845126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c35a1-86ff-4ef3-9463-a2c0c6311c5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:55.847087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c35a1-86ff-4ef3-9463-a2c0c6311c5c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:55.880327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1314/2619] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:39:55 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:39:55.884548 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac582b-eaae-4894-a1e3-20e79eb3f11a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:55.894856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac582b-eaae-4894-a1e3-20e79eb3f11a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:55.956433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3fb72-58f0-49fc-8425-238b29c027e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:55.957027 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3fb72-58f0-49fc-8425-238b29c027e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:55.957335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3fb72-58f0-49fc-8425-238b29c027e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:55.957591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3fb72-58f0-49fc-8425-238b29c027e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:55.958213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3fb72-58f0-49fc-8425-238b29c027e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:55.960222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3fb72-58f0-49fc-8425-238b29c027e5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:55.983137 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1306/2620] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:39:55 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:39:55.987353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c35a1-86ff-4ef3-9463-a2c0c6311c5c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:55.999151 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c35a1-86ff-4ef3-9463-a2c0c6311c5c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:56.003967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bfdd7-a007-43a6-b6f1-ad923c8d1ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:56.004571 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bfdd7-a007-43a6-b6f1-ad923c8d1ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:56.004849 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bfdd7-a007-43a6-b6f1-ad923c8d1ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:56.005118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bfdd7-a007-43a6-b6f1-ad923c8d1ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:56.005819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bfdd7-a007-43a6-b6f1-ad923c8d1ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:56.007401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bfdd7-a007-43a6-b6f1-ad923c8d1ca1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:56.017514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1315/2621] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:39:56.021646 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3fb72-58f0-49fc-8425-238b29c027e5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:56.031823 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3fb72-58f0-49fc-8425-238b29c027e5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:56.037293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fe569-d4a8-462c-aced-959bf646fecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:56.037918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fe569-d4a8-462c-aced-959bf646fecc admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:56.038155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fe569-d4a8-462c-aced-959bf646fecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:56.038447 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fe569-d4a8-462c-aced-959bf646fecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:56.039088 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fe569-d4a8-462c-aced-959bf646fecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:56.040679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fe569-d4a8-462c-aced-959bf646fecc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:56.051636 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1307/2622] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:39:56.055844 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bfdd7-a007-43a6-b6f1-ad923c8d1ca1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:56.065808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bfdd7-a007-43a6-b6f1-ad923c8d1ca1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:56.070716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e590bbe-6efe-4288-86cc-5af040dc6351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:56.071359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e590bbe-6efe-4288-86cc-5af040dc6351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:56.071636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e590bbe-6efe-4288-86cc-5af040dc6351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:56.071906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e590bbe-6efe-4288-86cc-5af040dc6351 admin admin] PATH_INFO: `/v3/projects/e4563767772c486f8fbb9ea131771412/users/a486a4bb6b724dacbb78a9e0fe6a789e/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:56.086146 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e590bbe-6efe-4288-86cc-5af040dc6351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4563767772c486f8fbb9ea131771412, user_id=a486a4bb6b724dacbb78a9e0fe6a789e, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:56.089111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e590bbe-6efe-4288-86cc-5af040dc6351 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:56.103445 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1316/2623] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:39:56 2022] PUT /identity/v3/projects/e4563767772c486f8fbb9ea131771412/users/a486a4bb6b724dacbb78a9e0fe6a789e/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:39:56.108569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0d162-0055-4931-8ba2-e041ecc63cb2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:56.108855 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0d162-0055-4931-8ba2-e041ecc63cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:56.109159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0d162-0055-4931-8ba2-e041ecc63cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:56.128268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-69b0d162-0055-4931-8ba2-e041ecc63cb2 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:39:56.129539 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-69b0d162-0055-4931-8ba2-e041ecc63cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:56.156934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:56.192159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1308/2624] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:39:56.202219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e590bbe-6efe-4288-86cc-5af040dc6351 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:39:56.203999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5e590bbe-6efe-4288-86cc-5af040dc6351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:39:56.239805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:56.264238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e590bbe-6efe-4288-86cc-5af040dc6351 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:39:56.268320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612b296-b5a4-427f-80c3-f8c478d22eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:39:56.268914 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612b296-b5a4-427f-80c3-f8c478d22eac service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:39:56.269225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612b296-b5a4-427f-80c3-f8c478d22eac service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:39:56.269495 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612b296-b5a4-427f-80c3-f8c478d22eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:39:56.278912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612b296-b5a4-427f-80c3-f8c478d22eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:39:56.280337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612b296-b5a4-427f-80c3-f8c478d22eac service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:39:56.289777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1317/2625] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:40:00.024603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0d162-0055-4931-8ba2-e041ecc63cb2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:00.026465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-69b0d162-0055-4931-8ba2-e041ecc63cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:00.043277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0d162-0055-4931-8ba2-e041ecc63cb2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:00.049420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:00.050279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:00.050718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:00.050993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] PATH_INFO: `/v3/users/70bb0b4438f845a998daeafb85504dec` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:00.054384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70bb0b4438f845a998daeafb85504dec)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:00.056317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:00.113557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70bb0b4438f845a998daeafb85504dec'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:00.123956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70bb0b4438f845a998daeafb85504dec'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:00.124707 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70bb0b4438f845a998daeafb85504dec'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:00.133299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70bb0b4438f845a998daeafb85504dec'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:00.146119 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70bb0b4438f845a998daeafb85504dec'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:00.155313 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1309/2626] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:40:00 2022] DELETE /identity/v3/users/70bb0b4438f845a998daeafb85504dec => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:00.157500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e49b5-e13e-40ad-87b5-14e6957e707f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:00.158010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e49b5-e13e-40ad-87b5-14e6957e707f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:00.158647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e49b5-e13e-40ad-87b5-14e6957e707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:00.178288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f40e49b5-e13e-40ad-87b5-14e6957e707f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:00.179703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f40e49b5-e13e-40ad-87b5-14e6957e707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:00.272822 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1318/2627] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:00.287840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:00.289461 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:00.375050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac9792-ae79-4c4d-8b22-1bc9ce04df25 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:00.378785 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe80a4-eb55-4477-bb1c-df94888a7a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:00.379412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe80a4-eb55-4477-bb1c-df94888a7a42 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:00.379692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe80a4-eb55-4477-bb1c-df94888a7a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:00.379961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe80a4-eb55-4477-bb1c-df94888a7a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:00.392210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe80a4-eb55-4477-bb1c-df94888a7a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:00.393836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe80a4-eb55-4477-bb1c-df94888a7a42 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:00.405933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1310/2628] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:00.454673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb1fc5-9f11-4093-899d-7d61c9893c67 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:00.455066 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb1fc5-9f11-4093-899d-7d61c9893c67 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:00.455215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb1fc5-9f11-4093-899d-7d61c9893c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:00.493478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-badb1fc5-9f11-4093-899d-7d61c9893c67 None None] MFA Rules not processed for user `a486a4bb6b724dacbb78a9e0fe6a789e`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:00.494922 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-badb1fc5-9f11-4093-899d-7d61c9893c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:00.574195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1319/2629] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:00.593853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe80a4-eb55-4477-bb1c-df94888a7a42 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:00.595848 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7bfe80a4-eb55-4477-bb1c-df94888a7a42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:00.687509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:00.723200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe80a4-eb55-4477-bb1c-df94888a7a42 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:00.728892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d31243-ea79-4785-8f30-e3639442aa02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:00.729535 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d31243-ea79-4785-8f30-e3639442aa02 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:00.729811 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d31243-ea79-4785-8f30-e3639442aa02 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:00.730083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d31243-ea79-4785-8f30-e3639442aa02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:00.747165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d31243-ea79-4785-8f30-e3639442aa02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:00.748771 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d31243-ea79-4785-8f30-e3639442aa02 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:00.757560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb1fc5-9f11-4093-899d-7d61c9893c67 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:00.759186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-badb1fc5-9f11-4093-899d-7d61c9893c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:00.759841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1311/2630] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:00.783386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb1fc5-9f11-4093-899d-7d61c9893c67 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:00.789179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:00.790105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:00.790548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:00.790960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc admin admin] PATH_INFO: `/v3/projects/f5eef26df9ed4b079274385534f4ec40` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:00.792326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5eef26df9ed4b079274385534f4ec40)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:00.794494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:00.855165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5eef26df9ed4b079274385534f4ec40', 'request_id': 'req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:00.866348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5eef26df9ed4b079274385534f4ec40', 'request_id': 'req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:00.871139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:00.873650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5eef26df9ed4b079274385534f4ec40', 'request_id': 'req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:00.892580 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc admin admin] The token cache is being invalidate because project f5eef26df9ed4b079274385534f4ec40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:40:00.892878 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:00.893712 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0eb3e418-cb15-4fc6-b6ea-ea814c022cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:00.895811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1320/2631] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:40:00 2022] DELETE /identity/v3/projects/f5eef26df9ed4b079274385534f4ec40 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:40:00.900838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b1360-0022-41c9-8b3a-d385f00c4243 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:00.901170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b1360-0022-41c9-8b3a-d385f00c4243 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:00.901427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b1360-0022-41c9-8b3a-d385f00c4243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:00.924324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5c7b1360-0022-41c9-8b3a-d385f00c4243 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:00.925721 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5c7b1360-0022-41c9-8b3a-d385f00c4243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:01.016652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1312/2632] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:01.025128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d919c8-c18f-4872-9d6b-6b6a04217044 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:01.025442 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d919c8-c18f-4872-9d6b-6b6a04217044 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:01.025716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d919c8-c18f-4872-9d6b-6b6a04217044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:01.048916 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-36d919c8-c18f-4872-9d6b-6b6a04217044 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:01.049575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-36d919c8-c18f-4872-9d6b-6b6a04217044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:01.059268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1321/2633] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:01.062335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b1360-0022-41c9-8b3a-d385f00c4243 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:01.072990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b1360-0022-41c9-8b3a-d385f00c4243 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:01.077877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527e5b4-5ab1-4ae2-9a7a-89109872a83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:01.078498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527e5b4-5ab1-4ae2-9a7a-89109872a83c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:01.078776 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527e5b4-5ab1-4ae2-9a7a-89109872a83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:01.079049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527e5b4-5ab1-4ae2-9a7a-89109872a83c admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:01.079705 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527e5b4-5ab1-4ae2-9a7a-89109872a83c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:40:01.080005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527e5b4-5ab1-4ae2-9a7a-89109872a83c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:01.081799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527e5b4-5ab1-4ae2-9a7a-89109872a83c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:01.095198 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1313/2634] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:40:01.099704 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d919c8-c18f-4872-9d6b-6b6a04217044 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:01.109197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d919c8-c18f-4872-9d6b-6b6a04217044 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:01.114125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe66d1-db44-42df-8377-77a115658f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:01.114789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe66d1-db44-42df-8377-77a115658f11 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:01.115086 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe66d1-db44-42df-8377-77a115658f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:01.115366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe66d1-db44-42df-8377-77a115658f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:01.116051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe66d1-db44-42df-8377-77a115658f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:01.118017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe66d1-db44-42df-8377-77a115658f11 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:01.160889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1322/2635] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:40:01 2022] POST /identity/v3/projects => generated 363 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:01.165378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527e5b4-5ab1-4ae2-9a7a-89109872a83c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:01.185778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527e5b4-5ab1-4ae2-9a7a-89109872a83c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:01.309585 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9fa5e-74a2-43ab-8ffc-577d5558d4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:01.310222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9fa5e-74a2-43ab-8ffc-577d5558d4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:01.313530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9fa5e-74a2-43ab-8ffc-577d5558d4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:01.313816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9fa5e-74a2-43ab-8ffc-577d5558d4f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:01.314568 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9fa5e-74a2-43ab-8ffc-577d5558d4f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:01.316532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9fa5e-74a2-43ab-8ffc-577d5558d4f5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:01.342735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1314/2636] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:40:01 2022] POST /identity/v3/users => generated 282 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:40:01.347392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe66d1-db44-42df-8377-77a115658f11 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:01.358424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe66d1-db44-42df-8377-77a115658f11 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:01.363399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9289961-4baf-4155-9c70-c86da1f9e3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:01.364078 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9289961-4baf-4155-9c70-c86da1f9e3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:01.364536 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9289961-4baf-4155-9c70-c86da1f9e3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:01.364812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9289961-4baf-4155-9c70-c86da1f9e3e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:01.365766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9289961-4baf-4155-9c70-c86da1f9e3e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:01.367238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9289961-4baf-4155-9c70-c86da1f9e3e3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:01.379921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1323/2637] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:40:01.383867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9fa5e-74a2-43ab-8ffc-577d5558d4f5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:01.394615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9fa5e-74a2-43ab-8ffc-577d5558d4f5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:01.401812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32499538-bef9-4042-a054-cdee2696f8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:01.402447 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32499538-bef9-4042-a054-cdee2696f8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:01.402720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32499538-bef9-4042-a054-cdee2696f8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:01.402990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32499538-bef9-4042-a054-cdee2696f8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:01.403641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32499538-bef9-4042-a054-cdee2696f8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:01.405254 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32499538-bef9-4042-a054-cdee2696f8a4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:01.417215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1315/2638] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:01.421416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9289961-4baf-4155-9c70-c86da1f9e3e3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:01.432532 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9289961-4baf-4155-9c70-c86da1f9e3e3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:01.439591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fc801-6b05-4c43-bc16-05e46d74a9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:01.440290 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fc801-6b05-4c43-bc16-05e46d74a9aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:01.440568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fc801-6b05-4c43-bc16-05e46d74a9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:01.441150 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fc801-6b05-4c43-bc16-05e46d74a9aa admin admin] PATH_INFO: `/v3/projects/f3893ecb6ef54acc9b0f71ff5ca9332b/users/5f81b3db5dc74039bf6547a0cc3a416a/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:01.457656 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fc801-6b05-4c43-bc16-05e46d74a9aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3893ecb6ef54acc9b0f71ff5ca9332b, user_id=5f81b3db5dc74039bf6547a0cc3a416a, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:01.460395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fc801-6b05-4c43-bc16-05e46d74a9aa admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:01.476537 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1324/2639] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:01 2022] PUT /identity/v3/projects/f3893ecb6ef54acc9b0f71ff5ca9332b/users/5f81b3db5dc74039bf6547a0cc3a416a/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:01.487392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f705e-cf3f-48bd-8c37-cb3f64e42c61 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:01.487866 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f705e-cf3f-48bd-8c37-cb3f64e42c61 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:01.488240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f705e-cf3f-48bd-8c37-cb3f64e42c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:01.519568 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b51f705e-cf3f-48bd-8c37-cb3f64e42c61 None None] MFA Rules not processed for user `5f81b3db5dc74039bf6547a0cc3a416a`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:01.521048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b51f705e-cf3f-48bd-8c37-cb3f64e42c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:01.548758 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:01.595641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1316/2640] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:01.600551 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fc801-6b05-4c43-bc16-05e46d74a9aa admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:01.666274 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fc801-6b05-4c43-bc16-05e46d74a9aa admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:01.674175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584478e8-101c-43ae-a960-c6fe88e0aea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:01.674800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584478e8-101c-43ae-a960-c6fe88e0aea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:01.675077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584478e8-101c-43ae-a960-c6fe88e0aea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:01.675522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584478e8-101c-43ae-a960-c6fe88e0aea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:01.676326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584478e8-101c-43ae-a960-c6fe88e0aea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:01.678472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584478e8-101c-43ae-a960-c6fe88e0aea9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:01.719973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1325/2641] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:40:01 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:01.723099 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f705e-cf3f-48bd-8c37-cb3f64e42c61 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:01.734670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f705e-cf3f-48bd-8c37-cb3f64e42c61 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:01.817333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea87fde-3154-46e8-ab57-73308469b986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:01.817887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea87fde-3154-46e8-ab57-73308469b986 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:01.818217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea87fde-3154-46e8-ab57-73308469b986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:01.818519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea87fde-3154-46e8-ab57-73308469b986 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:01.819164 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea87fde-3154-46e8-ab57-73308469b986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:01.821293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea87fde-3154-46e8-ab57-73308469b986 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:01.849475 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1317/2642] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:40:01 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:01.853299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584478e8-101c-43ae-a960-c6fe88e0aea9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:01.865223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584478e8-101c-43ae-a960-c6fe88e0aea9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:01.872034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ed5f0-ecae-4ad0-843b-3470d2974d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:01.872848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ed5f0-ecae-4ad0-843b-3470d2974d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:01.873314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ed5f0-ecae-4ad0-843b-3470d2974d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:01.873583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ed5f0-ecae-4ad0-843b-3470d2974d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:01.874401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ed5f0-ecae-4ad0-843b-3470d2974d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:01.876166 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ed5f0-ecae-4ad0-843b-3470d2974d7f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:01.888449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1326/2643] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:40:01.893582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea87fde-3154-46e8-ab57-73308469b986 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:01.905454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea87fde-3154-46e8-ab57-73308469b986 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:01.912609 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ef7a0-8941-405a-912b-afba734fcbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:01.913460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ef7a0-8941-405a-912b-afba734fcbc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:01.913847 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ef7a0-8941-405a-912b-afba734fcbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:01.914392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ef7a0-8941-405a-912b-afba734fcbc2 admin admin] PATH_INFO: `/v3/projects/448395c2190d4c8ba59d0478564d4dba/users/6838a00804b54137a1611206c91e32cc/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:01.944944 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ef7a0-8941-405a-912b-afba734fcbc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448395c2190d4c8ba59d0478564d4dba, user_id=6838a00804b54137a1611206c91e32cc, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:01.947809 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ef7a0-8941-405a-912b-afba734fcbc2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:01.966641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1318/2644] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:01 2022] PUT /identity/v3/projects/448395c2190d4c8ba59d0478564d4dba/users/6838a00804b54137a1611206c91e32cc/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:01.970690 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ed5f0-ecae-4ad0-843b-3470d2974d7f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:01.981119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ed5f0-ecae-4ad0-843b-3470d2974d7f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:02.053289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fbd4d-fcfe-4929-9f10-5fb2c53b8cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:02.053900 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fbd4d-fcfe-4929-9f10-5fb2c53b8cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:02.054172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fbd4d-fcfe-4929-9f10-5fb2c53b8cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:02.054438 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fbd4d-fcfe-4929-9f10-5fb2c53b8cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:02.055325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fbd4d-fcfe-4929-9f10-5fb2c53b8cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:02.056945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fbd4d-fcfe-4929-9f10-5fb2c53b8cb1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:02.058215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:02.070673 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1327/2645] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:02.074195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ef7a0-8941-405a-912b-afba734fcbc2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:02.084675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ef7a0-8941-405a-912b-afba734fcbc2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:02.089677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b986fb-605e-4276-b627-2d48f462c574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:02.090399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b986fb-605e-4276-b627-2d48f462c574 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:02.090754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b986fb-605e-4276-b627-2d48f462c574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:02.091071 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b986fb-605e-4276-b627-2d48f462c574 admin admin] PATH_INFO: `/v3/projects/448395c2190d4c8ba59d0478564d4dba/users/6838a00804b54137a1611206c91e32cc/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:02.093065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b986fb-605e-4276-b627-2d48f462c574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448395c2190d4c8ba59d0478564d4dba, user_id=6838a00804b54137a1611206c91e32cc, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:02.095744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b986fb-605e-4276-b627-2d48f462c574 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:02.106777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1319/2646] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:02 2022] PUT /identity/v3/projects/448395c2190d4c8ba59d0478564d4dba/users/6838a00804b54137a1611206c91e32cc/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:40:02.118687 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd33c5-3c69-4ff0-83e5-51e0a10f72b4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:02.119064 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd33c5-3c69-4ff0-83e5-51e0a10f72b4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:02.119397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd33c5-3c69-4ff0-83e5-51e0a10f72b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:02.148407 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e0bd33c5-3c69-4ff0-83e5-51e0a10f72b4 None None] MFA Rules not processed for user `6838a00804b54137a1611206c91e32cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:02.149698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e0bd33c5-3c69-4ff0-83e5-51e0a10f72b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:02.206114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5deb6-4b00-41a8-845d-9c0612a6b296 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:02.206668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5deb6-4b00-41a8-845d-9c0612a6b296 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:02.207144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5deb6-4b00-41a8-845d-9c0612a6b296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:02.227344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-54e5deb6-4b00-41a8-845d-9c0612a6b296 None None] MFA Rules not processed for user `a486a4bb6b724dacbb78a9e0fe6a789e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:02.228530 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-54e5deb6-4b00-41a8-845d-9c0612a6b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:02.232354 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1328/2647] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:02.244287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd33c5-3c69-4ff0-83e5-51e0a10f72b4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:02.245724 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e0bd33c5-3c69-4ff0-83e5-51e0a10f72b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:02.286836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1320/2648] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:02.301125 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd33c5-3c69-4ff0-83e5-51e0a10f72b4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:02.304708 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e5a19-a48f-4088-93ae-e56184189b34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:02.305397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e5a19-a48f-4088-93ae-e56184189b34 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:02.305623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e5a19-a48f-4088-93ae-e56184189b34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:02.305886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e5a19-a48f-4088-93ae-e56184189b34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:02.315373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e5a19-a48f-4088-93ae-e56184189b34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:02.316782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e5a19-a48f-4088-93ae-e56184189b34 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:02.327997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1329/2649] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:40:02.416307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5deb6-4b00-41a8-845d-9c0612a6b296 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:02.426559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5deb6-4b00-41a8-845d-9c0612a6b296 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:02.430617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8fd0b-874a-44d4-91f1-4db1477be1b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:02.431233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8fd0b-874a-44d4-91f1-4db1477be1b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:02.431502 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8fd0b-874a-44d4-91f1-4db1477be1b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:02.431763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8fd0b-874a-44d4-91f1-4db1477be1b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:02.458341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:02.501210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8fd0b-874a-44d4-91f1-4db1477be1b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:02.502681 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8fd0b-874a-44d4-91f1-4db1477be1b0 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:02.513258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1321/2650] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:02.978016 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e5a19-a48f-4088-93ae-e56184189b34 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:02.979501 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6d2e5a19-a48f-4088-93ae-e56184189b34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:03.064595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e5a19-a48f-4088-93ae-e56184189b34 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:03.069862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef5606-1de7-4367-b5ab-f4da0e13684d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:03.070473 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef5606-1de7-4367-b5ab-f4da0e13684d service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:03.070750 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef5606-1de7-4367-b5ab-f4da0e13684d service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:03.071019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef5606-1de7-4367-b5ab-f4da0e13684d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:03.072775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f3ef5606-1de7-4367-b5ab-f4da0e13684d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:03.084381 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef5606-1de7-4367-b5ab-f4da0e13684d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:03.086007 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef5606-1de7-4367-b5ab-f4da0e13684d service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:03.096985 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1330/2651] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:12.510655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=17 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:13.092505 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:14.150281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8fd0b-874a-44d4-91f1-4db1477be1b0 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:14.215085 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8fd0b-874a-44d4-91f1-4db1477be1b0 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:14.220222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:14.220859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:14.221143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:14.221434 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] PATH_INFO: `/v3/users/5f81b3db5dc74039bf6547a0cc3a416a` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:14.222628 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f81b3db5dc74039bf6547a0cc3a416a)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:14.224542 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:14.272326 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f81b3db5dc74039bf6547a0cc3a416a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:14.279851 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f81b3db5dc74039bf6547a0cc3a416a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:14.280520 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f81b3db5dc74039bf6547a0cc3a416a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:14.287575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f81b3db5dc74039bf6547a0cc3a416a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:14.300523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f81b3db5dc74039bf6547a0cc3a416a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:14.307073 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1322/2652] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:40:14 2022] DELETE /identity/v3/users/5f81b3db5dc74039bf6547a0cc3a416a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:14.311636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7547e-b06d-4526-bff3-20f53e106878 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:14.312113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7547e-b06d-4526-bff3-20f53e106878 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:14.312210 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7547e-b06d-4526-bff3-20f53e106878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:14.331179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3ae7547e-b06d-4526-bff3-20f53e106878 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:14.332375 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3ae7547e-b06d-4526-bff3-20f53e106878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:14.393533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1331/2653] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:14.403639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:14.405189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:14.464694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba2b75-a4c3-427f-b6c5-50f5bde43fc4 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:14.468352 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a1c5a-32db-4677-92da-bba1eaf91579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:14.468963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a1c5a-32db-4677-92da-bba1eaf91579 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:14.469267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a1c5a-32db-4677-92da-bba1eaf91579 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:14.469536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a1c5a-32db-4677-92da-bba1eaf91579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:14.478292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a1c5a-32db-4677-92da-bba1eaf91579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:14.479708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a1c5a-32db-4677-92da-bba1eaf91579 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:14.489370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1323/2654] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:14.779802 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7547e-b06d-4526-bff3-20f53e106878 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:14.781380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3ae7547e-b06d-4526-bff3-20f53e106878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:14.793967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7547e-b06d-4526-bff3-20f53e106878 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:14.799698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:14.800472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:14.800721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:14.801017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] PATH_INFO: `/v3/projects/f3893ecb6ef54acc9b0f71ff5ca9332b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:14.802243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3893ecb6ef54acc9b0f71ff5ca9332b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:14.804128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:14.854685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3893ecb6ef54acc9b0f71ff5ca9332b', 'request_id': 'req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:14.864156 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3893ecb6ef54acc9b0f71ff5ca9332b', 'request_id': 'req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:14.887316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3893ecb6ef54acc9b0f71ff5ca9332b', 'request_id': 'req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:14.906969 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] The token cache is being invalidate because project f3893ecb6ef54acc9b0f71ff5ca9332b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:40:14.907448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:14.908699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:14.911675 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1332/2655] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:40:14 2022] DELETE /identity/v3/projects/f3893ecb6ef54acc9b0f71ff5ca9332b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:14.916633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a1c5a-32db-4677-92da-bba1eaf91579 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:14.927316 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a1c5a-32db-4677-92da-bba1eaf91579 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:14.954689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:15.013584 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2fae1-6fc6-488e-ba03-77a551ad5681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:15.014234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2fae1-6fc6-488e-ba03-77a551ad5681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:15.014503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2fae1-6fc6-488e-ba03-77a551ad5681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:15.014762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2fae1-6fc6-488e-ba03-77a551ad5681 admin admin] PATH_INFO: `/v3/users/6838a00804b54137a1611206c91e32cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:15.015901 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2fae1-6fc6-488e-ba03-77a551ad5681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6838a00804b54137a1611206c91e32cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:15.017829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2fae1-6fc6-488e-ba03-77a551ad5681 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:15.069473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9dc2fae1-6fc6-488e-ba03-77a551ad5681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6838a00804b54137a1611206c91e32cc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:15.076837 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9dc2fae1-6fc6-488e-ba03-77a551ad5681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6838a00804b54137a1611206c91e32cc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:15.077471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9dc2fae1-6fc6-488e-ba03-77a551ad5681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6838a00804b54137a1611206c91e32cc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:15.084249 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9dc2fae1-6fc6-488e-ba03-77a551ad5681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6838a00804b54137a1611206c91e32cc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:15.095854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9dc2fae1-6fc6-488e-ba03-77a551ad5681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6838a00804b54137a1611206c91e32cc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:15.101543 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1324/2656] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:40:14 2022] DELETE /identity/v3/users/6838a00804b54137a1611206c91e32cc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:15.397523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:15.399098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:15.412621 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a2968-f62e-45b3-a359-7f8c1b1ca279 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:15.470297 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:15.470928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:15.471206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:15.471478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce admin admin] PATH_INFO: `/v3/projects/448395c2190d4c8ba59d0478564d4dba` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:15.472635 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=448395c2190d4c8ba59d0478564d4dba)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:15.474566 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:15.506311 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:15.523466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '448395c2190d4c8ba59d0478564d4dba', 'request_id': 'req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:15.530628 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '448395c2190d4c8ba59d0478564d4dba', 'request_id': 'req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:15.536230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '448395c2190d4c8ba59d0478564d4dba', 'request_id': 'req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:15.546715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce admin admin] The token cache is being invalidate because project 448395c2190d4c8ba59d0478564d4dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:40:15.546996 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:15.547611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9917fe9b-e32a-4da1-84af-7a5cbd32a0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:15.549060 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1333/2657] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:40:15 2022] DELETE /identity/v3/projects/448395c2190d4c8ba59d0478564d4dba => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:15.553983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c104327-0ac2-4ef4-ad8b-cd44744bc78f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:15.554478 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c104327-0ac2-4ef4-ad8b-cd44744bc78f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:15.554600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c104327-0ac2-4ef4-ad8b-cd44744bc78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:15.574407 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5c104327-0ac2-4ef4-ad8b-cd44744bc78f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:15.575629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5c104327-0ac2-4ef4-ad8b-cd44744bc78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:15.641183 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1325/2658] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:15.649593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cec99-1364-4814-8f86-73c2a8df9f20 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:15.649876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cec99-1364-4814-8f86-73c2a8df9f20 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:15.650143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cec99-1364-4814-8f86-73c2a8df9f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:15.672991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ff2cec99-1364-4814-8f86-73c2a8df9f20 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:15.674250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ff2cec99-1364-4814-8f86-73c2a8df9f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:15.682818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1334/2659] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:15.687443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c104327-0ac2-4ef4-ad8b-cd44744bc78f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:15.697656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c104327-0ac2-4ef4-ad8b-cd44744bc78f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:15.702654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1a4cf-12d2-4457-9711-925646b1211b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:15.703257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1a4cf-12d2-4457-9711-925646b1211b admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:15.703519 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1a4cf-12d2-4457-9711-925646b1211b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:15.703800 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1a4cf-12d2-4457-9711-925646b1211b admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:15.704418 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1a4cf-12d2-4457-9711-925646b1211b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:40:15.704715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1a4cf-12d2-4457-9711-925646b1211b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:15.706351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1a4cf-12d2-4457-9711-925646b1211b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:15.718488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1326/2660] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:40:15.722855 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cec99-1364-4814-8f86-73c2a8df9f20 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:15.732262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cec99-1364-4814-8f86-73c2a8df9f20 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:15.737773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33043286-204b-4560-a28b-94965a2846b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:15.738366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33043286-204b-4560-a28b-94965a2846b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:15.738642 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33043286-204b-4560-a28b-94965a2846b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:15.738907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33043286-204b-4560-a28b-94965a2846b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:15.739530 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33043286-204b-4560-a28b-94965a2846b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:15.741471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33043286-204b-4560-a28b-94965a2846b1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:15.774229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1335/2661] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:40:15 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:15.778572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1a4cf-12d2-4457-9711-925646b1211b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:15.791420 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1a4cf-12d2-4457-9711-925646b1211b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:15.818315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:15.851708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524a2a3-0059-469d-b403-130acab8e121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:15.852313 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524a2a3-0059-469d-b403-130acab8e121 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:15.852588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524a2a3-0059-469d-b403-130acab8e121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:15.852938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524a2a3-0059-469d-b403-130acab8e121 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:15.853593 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524a2a3-0059-469d-b403-130acab8e121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:15.855479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524a2a3-0059-469d-b403-130acab8e121 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:15.877735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1327/2662] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:40:15 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:15.882118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33043286-204b-4560-a28b-94965a2846b1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:15.892685 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33043286-204b-4560-a28b-94965a2846b1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:15.897853 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec512251-408e-4620-980a-c055935e4f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:15.898444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec512251-408e-4620-980a-c055935e4f23 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:15.898723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec512251-408e-4620-980a-c055935e4f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:15.898982 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec512251-408e-4620-980a-c055935e4f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:15.899616 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec512251-408e-4620-980a-c055935e4f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:15.901246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec512251-408e-4620-980a-c055935e4f23 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:15.911813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1336/2663] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:15.915782 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524a2a3-0059-469d-b403-130acab8e121 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:15.925868 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524a2a3-0059-469d-b403-130acab8e121 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:15.931509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b4c4e-c200-4c9a-afb3-536596736552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:15.932104 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b4c4e-c200-4c9a-afb3-536596736552 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:15.932372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b4c4e-c200-4c9a-afb3-536596736552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:15.932634 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b4c4e-c200-4c9a-afb3-536596736552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:15.933316 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b4c4e-c200-4c9a-afb3-536596736552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:15.934902 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b4c4e-c200-4c9a-afb3-536596736552 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:15.945789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1328/2664] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:40:15.950098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec512251-408e-4620-980a-c055935e4f23 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:15.960106 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec512251-408e-4620-980a-c055935e4f23 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:15.965380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373a1a-44ab-4c9f-8dbf-178c81c7fc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:15.966096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373a1a-44ab-4c9f-8dbf-178c81c7fc5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:15.966346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373a1a-44ab-4c9f-8dbf-178c81c7fc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:15.966614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373a1a-44ab-4c9f-8dbf-178c81c7fc5a admin admin] PATH_INFO: `/v3/projects/f370f5e987304c17a05a105950b5a597/users/6aded94116304276a21a1cab4d494909/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:15.981227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a373a1a-44ab-4c9f-8dbf-178c81c7fc5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f370f5e987304c17a05a105950b5a597, user_id=6aded94116304276a21a1cab4d494909, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:15.983843 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a373a1a-44ab-4c9f-8dbf-178c81c7fc5a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:15.997935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1337/2665] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:15 2022] PUT /identity/v3/projects/f370f5e987304c17a05a105950b5a597/users/6aded94116304276a21a1cab4d494909/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:16.008262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52eece-f54a-4d9c-b712-e2f2ef4786b3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:16.008584 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52eece-f54a-4d9c-b712-e2f2ef4786b3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:16.008858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52eece-f54a-4d9c-b712-e2f2ef4786b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:16.037463 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ad52eece-f54a-4d9c-b712-e2f2ef4786b3 None None] MFA Rules not processed for user `6aded94116304276a21a1cab4d494909`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:16.038987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ad52eece-f54a-4d9c-b712-e2f2ef4786b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:16.108701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1329/2666] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:16.113353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373a1a-44ab-4c9f-8dbf-178c81c7fc5a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:16.178233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373a1a-44ab-4c9f-8dbf-178c81c7fc5a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:16.183531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614ae15-af42-40c5-83db-c5a01c858e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:16.184170 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614ae15-af42-40c5-83db-c5a01c858e22 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:16.184440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614ae15-af42-40c5-83db-c5a01c858e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:16.184701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614ae15-af42-40c5-83db-c5a01c858e22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:16.185369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614ae15-af42-40c5-83db-c5a01c858e22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:16.187282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614ae15-af42-40c5-83db-c5a01c858e22 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:16.218881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1338/2667] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:40:16 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:16.222926 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52eece-f54a-4d9c-b712-e2f2ef4786b3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:16.234794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52eece-f54a-4d9c-b712-e2f2ef4786b3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:16.297312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f04dd1-3d23-4090-a482-db1b5e13d7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:16.298917 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f04dd1-3d23-4090-a482-db1b5e13d7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:16.298917 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f04dd1-3d23-4090-a482-db1b5e13d7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:16.298917 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f04dd1-3d23-4090-a482-db1b5e13d7a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:16.299223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f04dd1-3d23-4090-a482-db1b5e13d7a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:16.301287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f04dd1-3d23-4090-a482-db1b5e13d7a4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:16.323473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1330/2668] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:40:16 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:40:16.327494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614ae15-af42-40c5-83db-c5a01c858e22 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:16.337660 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614ae15-af42-40c5-83db-c5a01c858e22 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:16.342590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17fc97-7f96-47ed-a7ad-c3cbab0818e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:16.343197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17fc97-7f96-47ed-a7ad-c3cbab0818e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:16.343470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17fc97-7f96-47ed-a7ad-c3cbab0818e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:16.343736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17fc97-7f96-47ed-a7ad-c3cbab0818e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:16.344380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17fc97-7f96-47ed-a7ad-c3cbab0818e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:16.346010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17fc97-7f96-47ed-a7ad-c3cbab0818e2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:16.357067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1339/2669] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:16.361388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f04dd1-3d23-4090-a482-db1b5e13d7a4 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:16.371653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f04dd1-3d23-4090-a482-db1b5e13d7a4 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:16.376972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e9ae8-49db-4f3a-af56-de43b452939b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:16.377727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e9ae8-49db-4f3a-af56-de43b452939b admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:16.377915 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e9ae8-49db-4f3a-af56-de43b452939b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:16.378188 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e9ae8-49db-4f3a-af56-de43b452939b admin admin] PATH_INFO: `/v3/projects/064a0b701d1a46c9b1271b6119097eac/users/7063737de043463a8a398ef1ed6ee25f/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:16.394155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e9ae8-49db-4f3a-af56-de43b452939b admin admin] RBAC: Authorizing `identity:create_grant(project_id=064a0b701d1a46c9b1271b6119097eac, user_id=7063737de043463a8a398ef1ed6ee25f, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:16.396927 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e9ae8-49db-4f3a-af56-de43b452939b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:16.407191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17fc97-7f96-47ed-a7ad-c3cbab0818e2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:16.410287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0d17fc97-7f96-47ed-a7ad-c3cbab0818e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:16.413863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1331/2670] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:16 2022] PUT /identity/v3/projects/064a0b701d1a46c9b1271b6119097eac/users/7063737de043463a8a398ef1ed6ee25f/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:40:16.418480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e9ae8-49db-4f3a-af56-de43b452939b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:16.423229 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:16.442390 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:16.492016 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17fc97-7f96-47ed-a7ad-c3cbab0818e2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:16.500004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e9ae8-49db-4f3a-af56-de43b452939b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:16.504789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3b29e-8836-4ece-bb23-1db83f245279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:16.505452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3b29e-8836-4ece-bb23-1db83f245279 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:16.505733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3b29e-8836-4ece-bb23-1db83f245279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:16.506031 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3b29e-8836-4ece-bb23-1db83f245279 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:16.506731 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3b29e-8836-4ece-bb23-1db83f245279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:16.508346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3b29e-8836-4ece-bb23-1db83f245279 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:16.518218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1332/2671] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:40:16.522276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3b29e-8836-4ece-bb23-1db83f245279 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:16.531873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3b29e-8836-4ece-bb23-1db83f245279 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:16.536457 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fa3c5-6288-41e2-9adf-48525dc54c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:16.537108 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fa3c5-6288-41e2-9adf-48525dc54c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:16.537395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fa3c5-6288-41e2-9adf-48525dc54c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:16.537666 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fa3c5-6288-41e2-9adf-48525dc54c10 admin admin] PATH_INFO: `/v3/projects/064a0b701d1a46c9b1271b6119097eac/users/7063737de043463a8a398ef1ed6ee25f/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:16.539600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fa3c5-6288-41e2-9adf-48525dc54c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064a0b701d1a46c9b1271b6119097eac, user_id=7063737de043463a8a398ef1ed6ee25f, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:16.542372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fa3c5-6288-41e2-9adf-48525dc54c10 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:16.552423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1333/2672] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:16 2022] PUT /identity/v3/projects/064a0b701d1a46c9b1271b6119097eac/users/7063737de043463a8a398ef1ed6ee25f/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:40:16.554073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8c45d-2d9a-4b2d-890c-dae0e9d5ff04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:16.554688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8c45d-2d9a-4b2d-890c-dae0e9d5ff04 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:16.554964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8c45d-2d9a-4b2d-890c-dae0e9d5ff04 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:16.555233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8c45d-2d9a-4b2d-890c-dae0e9d5ff04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:16.557062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5cd8c45d-2d9a-4b2d-890c-dae0e9d5ff04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:16.563809 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d8c93-a0ad-4857-90dd-8e90f8b84fe1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:16.564283 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d8c93-a0ad-4857-90dd-8e90f8b84fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:16.564403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d8c93-a0ad-4857-90dd-8e90f8b84fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:16.569638 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8c45d-2d9a-4b2d-890c-dae0e9d5ff04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:16.571247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8c45d-2d9a-4b2d-890c-dae0e9d5ff04 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:16.582013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1340/2673] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:40:16.592901 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-030d8c93-a0ad-4857-90dd-8e90f8b84fe1 None None] MFA Rules not processed for user `7063737de043463a8a398ef1ed6ee25f`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:16.594145 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-030d8c93-a0ad-4857-90dd-8e90f8b84fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:16.668957 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1334/2674] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:40:16.683972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8c45d-2d9a-4b2d-890c-dae0e9d5ff04 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:16.685548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5cd8c45d-2d9a-4b2d-890c-dae0e9d5ff04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:16.742584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8c45d-2d9a-4b2d-890c-dae0e9d5ff04 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:16.793579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a8a05-8144-4dde-86e7-7f6bdf5de13b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:16.794206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a8a05-8144-4dde-86e7-7f6bdf5de13b service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:16.794483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a8a05-8144-4dde-86e7-7f6bdf5de13b service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:16.794756 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a8a05-8144-4dde-86e7-7f6bdf5de13b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:16.803979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a8a05-8144-4dde-86e7-7f6bdf5de13b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:16.805493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a8a05-8144-4dde-86e7-7f6bdf5de13b service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:16.815721 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1341/2675] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:40:16.893354 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d8c93-a0ad-4857-90dd-8e90f8b84fe1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:16.903615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d8c93-a0ad-4857-90dd-8e90f8b84fe1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:16.907465 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab1b02-2ba3-4711-b3c2-369082593dff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:16.908065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab1b02-2ba3-4711-b3c2-369082593dff service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:16.908370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab1b02-2ba3-4711-b3c2-369082593dff service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:16.908644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab1b02-2ba3-4711-b3c2-369082593dff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:16.963810 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab1b02-2ba3-4711-b3c2-369082593dff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:16.965286 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab1b02-2ba3-4711-b3c2-369082593dff service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:16.974494 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1335/2676] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:19.749159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a8a05-8144-4dde-86e7-7f6bdf5de13b service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:19.758585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:19.815524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a8a05-8144-4dde-86e7-7f6bdf5de13b service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:19.820468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:19.821152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:19.821457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:19.821699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] PATH_INFO: `/v3/users/6aded94116304276a21a1cab4d494909` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:19.822866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aded94116304276a21a1cab4d494909)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:19.824776 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:19.875714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aded94116304276a21a1cab4d494909'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:19.883943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aded94116304276a21a1cab4d494909'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:19.884659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aded94116304276a21a1cab4d494909'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:19.891581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aded94116304276a21a1cab4d494909'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:19.904204 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aded94116304276a21a1cab4d494909'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:19.910804 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1342/2677] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:40:19 2022] DELETE /identity/v3/users/6aded94116304276a21a1cab4d494909 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:19.915513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43243a-306e-43aa-a617-37ade8b2d323 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:19.916021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43243a-306e-43aa-a617-37ade8b2d323 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:19.916121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43243a-306e-43aa-a617-37ade8b2d323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:19.942877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4e43243a-306e-43aa-a617-37ade8b2d323 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:19.944269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4e43243a-306e-43aa-a617-37ade8b2d323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:19.948688 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:19.965069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:19.966532 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:20.020190 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1336/2678] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:20.030317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43243a-306e-43aa-a617-37ade8b2d323 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:20.031938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4e43243a-306e-43aa-a617-37ade8b2d323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:20.055193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f5ba4-b1e8-46c4-9d07-fc53aaad922f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:20.063265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71857d47-8a84-4866-b8eb-4e4e866eda13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:20.063906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71857d47-8a84-4866-b8eb-4e4e866eda13 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:20.064182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71857d47-8a84-4866-b8eb-4e4e866eda13 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:20.064451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71857d47-8a84-4866-b8eb-4e4e866eda13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:20.069419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-71857d47-8a84-4866-b8eb-4e4e866eda13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:20.087813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71857d47-8a84-4866-b8eb-4e4e866eda13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:20.090426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71857d47-8a84-4866-b8eb-4e4e866eda13 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:20.103669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1343/2679] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:40:20.108647 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43243a-306e-43aa-a617-37ade8b2d323 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:20.112737 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d6f6a-c562-4983-a824-78f2dcc025e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:20.113522 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d6f6a-c562-4983-a824-78f2dcc025e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:20.113901 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d6f6a-c562-4983-a824-78f2dcc025e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:20.114271 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d6f6a-c562-4983-a824-78f2dcc025e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:20.125245 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d6f6a-c562-4983-a824-78f2dcc025e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:20.127570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d6f6a-c562-4983-a824-78f2dcc025e3 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:20.139089 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1337/2680] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:40:20.543318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71857d47-8a84-4866-b8eb-4e4e866eda13 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:20.544800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-71857d47-8a84-4866-b8eb-4e4e866eda13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:20.571816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71857d47-8a84-4866-b8eb-4e4e866eda13 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:20.577803 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:20.578706 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:20.579781 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:20.580251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] PATH_INFO: `/v3/projects/f370f5e987304c17a05a105950b5a597` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:20.582082 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f370f5e987304c17a05a105950b5a597)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:20.584812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:20.596697 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:20.649222 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f370f5e987304c17a05a105950b5a597', 'request_id': 'req-f23bf0f7-e722-4b96-86f9-e741f16d1284'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:20.659462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f370f5e987304c17a05a105950b5a597', 'request_id': 'req-f23bf0f7-e722-4b96-86f9-e741f16d1284'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:20.668169 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f370f5e987304c17a05a105950b5a597', 'request_id': 'req-f23bf0f7-e722-4b96-86f9-e741f16d1284'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:20.682263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] The token cache is being invalidate because project f370f5e987304c17a05a105950b5a597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:40:20.682759 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:20.683426 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:20.686128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1344/2681] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:40:20 2022] DELETE /identity/v3/projects/f370f5e987304c17a05a105950b5a597 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:40:20.689130 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d6f6a-c562-4983-a824-78f2dcc025e3 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:20.700615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d6f6a-c562-4983-a824-78f2dcc025e3 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:20.764660 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52a5b6-3678-4af6-b58f-afa82a5eaf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:20.765342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a5b6-3678-4af6-b58f-afa82a5eaf1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:20.765624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a5b6-3678-4af6-b58f-afa82a5eaf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:20.766051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a5b6-3678-4af6-b58f-afa82a5eaf1a admin admin] PATH_INFO: `/v3/users/7063737de043463a8a398ef1ed6ee25f` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:20.767212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a5b6-3678-4af6-b58f-afa82a5eaf1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7063737de043463a8a398ef1ed6ee25f)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:20.769287 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a5b6-3678-4af6-b58f-afa82a5eaf1a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:20.818615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4e52a5b6-3678-4af6-b58f-afa82a5eaf1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7063737de043463a8a398ef1ed6ee25f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:20.827971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4e52a5b6-3678-4af6-b58f-afa82a5eaf1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7063737de043463a8a398ef1ed6ee25f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:20.828812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4e52a5b6-3678-4af6-b58f-afa82a5eaf1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7063737de043463a8a398ef1ed6ee25f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:20.830396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:20.841206 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4e52a5b6-3678-4af6-b58f-afa82a5eaf1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7063737de043463a8a398ef1ed6ee25f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:20.855301 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4e52a5b6-3678-4af6-b58f-afa82a5eaf1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7063737de043463a8a398ef1ed6ee25f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:20.862629 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1338/2682] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:40:20 2022] DELETE /identity/v3/users/7063737de043463a8a398ef1ed6ee25f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:21.152683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:21.154369 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:21.178262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bf0f7-e722-4b96-86f9-e741f16d1284 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:21.277639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:21.278267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:21.278561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:21.278824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f admin admin] PATH_INFO: `/v3/projects/064a0b701d1a46c9b1271b6119097eac` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:21.280045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f admin admin] RBAC: Authorizing `identity:delete_project(project_id=064a0b701d1a46c9b1271b6119097eac)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:21.282000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:21.335250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064a0b701d1a46c9b1271b6119097eac', 'request_id': 'req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:21.343454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064a0b701d1a46c9b1271b6119097eac', 'request_id': 'req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:21.349140 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064a0b701d1a46c9b1271b6119097eac', 'request_id': 'req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:21.360084 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f admin admin] The token cache is being invalidate because project 064a0b701d1a46c9b1271b6119097eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:40:21.360366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:21.360981 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-18aa56a6-cebd-4c29-aedb-df8f6a9db20f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:21.362490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1345/2683] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:40:21 2022] DELETE /identity/v3/projects/064a0b701d1a46c9b1271b6119097eac => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:21.367772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb273d86-cede-4daf-aee9-2da45d1122dc None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:21.368305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb273d86-cede-4daf-aee9-2da45d1122dc None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:21.368305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb273d86-cede-4daf-aee9-2da45d1122dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:21.389299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-eb273d86-cede-4daf-aee9-2da45d1122dc None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:21.390499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-eb273d86-cede-4daf-aee9-2da45d1122dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:21.450208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1339/2684] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:21.458489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25478376-1820-443c-be26-e10a7dbac6e6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:21.458770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25478376-1820-443c-be26-e10a7dbac6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:21.459033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25478376-1820-443c-be26-e10a7dbac6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:21.480664 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-25478376-1820-443c-be26-e10a7dbac6e6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:21.481878 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-25478376-1820-443c-be26-e10a7dbac6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:21.490729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1346/2685] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:21.495230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb273d86-cede-4daf-aee9-2da45d1122dc None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:21.505206 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb273d86-cede-4daf-aee9-2da45d1122dc None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:21.510603 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03765c71-4a8b-4d84-97e9-934fd1c21a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:21.511201 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03765c71-4a8b-4d84-97e9-934fd1c21a51 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:21.511474 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03765c71-4a8b-4d84-97e9-934fd1c21a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:21.511743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03765c71-4a8b-4d84-97e9-934fd1c21a51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:21.512384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03765c71-4a8b-4d84-97e9-934fd1c21a51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:40:21.512679 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03765c71-4a8b-4d84-97e9-934fd1c21a51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:21.514309 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03765c71-4a8b-4d84-97e9-934fd1c21a51 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:21.526067 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1340/2686] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:40:21.530425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25478376-1820-443c-be26-e10a7dbac6e6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:21.539950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25478376-1820-443c-be26-e10a7dbac6e6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:21.546140 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a3889-e7c8-4189-af46-834f4aa4f868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:21.546731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a3889-e7c8-4189-af46-834f4aa4f868 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:21.546993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a3889-e7c8-4189-af46-834f4aa4f868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:21.547250 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a3889-e7c8-4189-af46-834f4aa4f868 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:21.547865 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a3889-e7c8-4189-af46-834f4aa4f868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:21.549780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a3889-e7c8-4189-af46-834f4aa4f868 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:21.566508 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:21.586531 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1347/2687] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:40:21 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:21.591077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03765c71-4a8b-4d84-97e9-934fd1c21a51 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:21.601821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03765c71-4a8b-4d84-97e9-934fd1c21a51 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:21.659936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c492ca2-4775-4666-9b88-1b15ad5ed045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:21.660557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c492ca2-4775-4666-9b88-1b15ad5ed045 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:21.660839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c492ca2-4775-4666-9b88-1b15ad5ed045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:21.661133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c492ca2-4775-4666-9b88-1b15ad5ed045 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:21.661778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c492ca2-4775-4666-9b88-1b15ad5ed045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:21.663731 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c492ca2-4775-4666-9b88-1b15ad5ed045 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:21.685453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1341/2688] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:40:21 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:21.689687 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a3889-e7c8-4189-af46-834f4aa4f868 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:21.700155 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a3889-e7c8-4189-af46-834f4aa4f868 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:21.705073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1082a-5649-460b-9450-e0d48b1981b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:21.705699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1082a-5649-460b-9450-e0d48b1981b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:21.705977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1082a-5649-460b-9450-e0d48b1981b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:21.706244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1082a-5649-460b-9450-e0d48b1981b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:21.706881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1082a-5649-460b-9450-e0d48b1981b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:21.708479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1082a-5649-460b-9450-e0d48b1981b4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:21.719172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1348/2689] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:21.723306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c492ca2-4775-4666-9b88-1b15ad5ed045 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:21.734079 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c492ca2-4775-4666-9b88-1b15ad5ed045 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:21.739826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98624f4-a462-4778-bf4b-b622f83585ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:21.740455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98624f4-a462-4778-bf4b-b622f83585ec admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:21.740732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98624f4-a462-4778-bf4b-b622f83585ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:21.741001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98624f4-a462-4778-bf4b-b622f83585ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:21.741688 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98624f4-a462-4778-bf4b-b622f83585ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:21.743258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98624f4-a462-4778-bf4b-b622f83585ec admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:21.756387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1342/2690] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:21.760817 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1082a-5649-460b-9450-e0d48b1981b4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:21.771069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1082a-5649-460b-9450-e0d48b1981b4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:21.776989 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8eab56-fe3b-438f-9bc6-e60ea67a072d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:21.777664 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8eab56-fe3b-438f-9bc6-e60ea67a072d admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:21.777953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8eab56-fe3b-438f-9bc6-e60ea67a072d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:21.778221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8eab56-fe3b-438f-9bc6-e60ea67a072d admin admin] PATH_INFO: `/v3/projects/0827222448714e309da5d8ce8e2292f6/users/e1ae809791f34eb3b9d88033d984ba0a/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:21.793865 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8eab56-fe3b-438f-9bc6-e60ea67a072d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0827222448714e309da5d8ce8e2292f6, user_id=e1ae809791f34eb3b9d88033d984ba0a, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:21.796496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8eab56-fe3b-438f-9bc6-e60ea67a072d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:21.812588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1349/2691] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:21 2022] PUT /identity/v3/projects/0827222448714e309da5d8ce8e2292f6/users/e1ae809791f34eb3b9d88033d984ba0a/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:21.824716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddacfe-fb0c-465e-b8f8-dfb36e4b4866 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:21.825000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddacfe-fb0c-465e-b8f8-dfb36e4b4866 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:21.825303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddacfe-fb0c-465e-b8f8-dfb36e4b4866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:21.855216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-69ddacfe-fb0c-465e-b8f8-dfb36e4b4866 None None] MFA Rules not processed for user `e1ae809791f34eb3b9d88033d984ba0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:21.856379 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-69ddacfe-fb0c-465e-b8f8-dfb36e4b4866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:21.884947 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:21.925169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1343/2692] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:21.929899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8eab56-fe3b-438f-9bc6-e60ea67a072d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:21.996228 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8eab56-fe3b-438f-9bc6-e60ea67a072d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:22.001259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90064353-34eb-46b3-a730-de34e12264dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:22.001861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90064353-34eb-46b3-a730-de34e12264dd admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:22.002131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90064353-34eb-46b3-a730-de34e12264dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:22.002395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90064353-34eb-46b3-a730-de34e12264dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:22.003017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90064353-34eb-46b3-a730-de34e12264dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:22.004891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90064353-34eb-46b3-a730-de34e12264dd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:22.032253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1350/2693] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:40:21 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:22.036746 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddacfe-fb0c-465e-b8f8-dfb36e4b4866 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:22.048546 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddacfe-fb0c-465e-b8f8-dfb36e4b4866 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:22.108402 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec1f5c-dcd9-44c8-aba8-12bc9d27fc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:22.108928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec1f5c-dcd9-44c8-aba8-12bc9d27fc38 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:22.109240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec1f5c-dcd9-44c8-aba8-12bc9d27fc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:22.109908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec1f5c-dcd9-44c8-aba8-12bc9d27fc38 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:22.110799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec1f5c-dcd9-44c8-aba8-12bc9d27fc38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:22.112716 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec1f5c-dcd9-44c8-aba8-12bc9d27fc38 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:22.135126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1344/2694] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:40:22 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:40:22.139607 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90064353-34eb-46b3-a730-de34e12264dd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:22.150302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90064353-34eb-46b3-a730-de34e12264dd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:22.155462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43bd14-086d-400e-a733-718105af9252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:22.156095 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43bd14-086d-400e-a733-718105af9252 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:22.156352 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43bd14-086d-400e-a733-718105af9252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:22.156619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43bd14-086d-400e-a733-718105af9252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:22.157311 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43bd14-086d-400e-a733-718105af9252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:22.158910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43bd14-086d-400e-a733-718105af9252 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:22.169598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1351/2695] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:22.173857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec1f5c-dcd9-44c8-aba8-12bc9d27fc38 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:22.184013 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec1f5c-dcd9-44c8-aba8-12bc9d27fc38 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:22.189264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76fb9f-3808-42d2-8a0e-ddecba4db5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:22.189908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76fb9f-3808-42d2-8a0e-ddecba4db5d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:22.190188 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76fb9f-3808-42d2-8a0e-ddecba4db5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:22.190487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76fb9f-3808-42d2-8a0e-ddecba4db5d2 admin admin] PATH_INFO: `/v3/projects/cafca35e180d4a43ad474e3e2ea61800/users/cf75cb4f56794589ae7af00386bbea0c/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:22.206351 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76fb9f-3808-42d2-8a0e-ddecba4db5d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cafca35e180d4a43ad474e3e2ea61800, user_id=cf75cb4f56794589ae7af00386bbea0c, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:22.208939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76fb9f-3808-42d2-8a0e-ddecba4db5d2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:22.224940 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1345/2696] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:22 2022] PUT /identity/v3/projects/cafca35e180d4a43ad474e3e2ea61800/users/cf75cb4f56794589ae7af00386bbea0c/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:22.229006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43bd14-086d-400e-a733-718105af9252 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:22.239002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43bd14-086d-400e-a733-718105af9252 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:22.298363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb2369-407d-4cd3-bb95-85ce90c6740e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:22.298961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb2369-407d-4cd3-bb95-85ce90c6740e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:22.299230 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb2369-407d-4cd3-bb95-85ce90c6740e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:22.299490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb2369-407d-4cd3-bb95-85ce90c6740e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:22.300118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb2369-407d-4cd3-bb95-85ce90c6740e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:22.301751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb2369-407d-4cd3-bb95-85ce90c6740e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:22.311330 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1352/2697] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:22.315674 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76fb9f-3808-42d2-8a0e-ddecba4db5d2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:22.325872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76fb9f-3808-42d2-8a0e-ddecba4db5d2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:22.330816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a1f6d-fdf0-47ca-b2d1-911369735412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:22.331454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a1f6d-fdf0-47ca-b2d1-911369735412 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:22.331728 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a1f6d-fdf0-47ca-b2d1-911369735412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:22.331987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a1f6d-fdf0-47ca-b2d1-911369735412 admin admin] PATH_INFO: `/v3/projects/cafca35e180d4a43ad474e3e2ea61800/users/cf75cb4f56794589ae7af00386bbea0c/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:22.334028 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a1f6d-fdf0-47ca-b2d1-911369735412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cafca35e180d4a43ad474e3e2ea61800, user_id=cf75cb4f56794589ae7af00386bbea0c, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:22.336615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a1f6d-fdf0-47ca-b2d1-911369735412 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:22.347237 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1346/2698] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:22 2022] PUT /identity/v3/projects/cafca35e180d4a43ad474e3e2ea61800/users/cf75cb4f56794589ae7af00386bbea0c/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:40:22.357406 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd189a6-8660-4f03-99f8-409d8823fa3d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:22.357687 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd189a6-8660-4f03-99f8-409d8823fa3d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:22.357952 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd189a6-8660-4f03-99f8-409d8823fa3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:22.386582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3fd189a6-8660-4f03-99f8-409d8823fa3d None None] MFA Rules not processed for user `cf75cb4f56794589ae7af00386bbea0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:22.387716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3fd189a6-8660-4f03-99f8-409d8823fa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:22.406444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:22.458172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1353/2699] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:22.470828 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a1f6d-fdf0-47ca-b2d1-911369735412 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:22.472576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b33a1f6d-fdf0-47ca-b2d1-911369735412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:22.532334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a1f6d-fdf0-47ca-b2d1-911369735412 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:22.536369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0ba94-d308-41ea-a986-5bac8dfd70d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:22.536987 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0ba94-d308-41ea-a986-5bac8dfd70d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:22.537296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0ba94-d308-41ea-a986-5bac8dfd70d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:22.537566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0ba94-d308-41ea-a986-5bac8dfd70d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:22.546979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0ba94-d308-41ea-a986-5bac8dfd70d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:22.548408 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0ba94-d308-41ea-a986-5bac8dfd70d3 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:22.558330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1347/2700] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:22.627041 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd189a6-8660-4f03-99f8-409d8823fa3d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:22.691937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd189a6-8660-4f03-99f8-409d8823fa3d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:22.697031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:22.697699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:22.697990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:22.698260 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] PATH_INFO: `/v3/users/e1ae809791f34eb3b9d88033d984ba0a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:22.699489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1ae809791f34eb3b9d88033d984ba0a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:22.701451 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:22.751419 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1ae809791f34eb3b9d88033d984ba0a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:22.759736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1ae809791f34eb3b9d88033d984ba0a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:22.760549 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1ae809791f34eb3b9d88033d984ba0a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:22.768496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1ae809791f34eb3b9d88033d984ba0a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:22.782130 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1ae809791f34eb3b9d88033d984ba0a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:22.788136 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1354/2701] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:40:22 2022] DELETE /identity/v3/users/e1ae809791f34eb3b9d88033d984ba0a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:22.792974 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1b6ca-1c03-4877-9e0b-62c584c0d6d5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:22.793372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1b6ca-1c03-4877-9e0b-62c584c0d6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:22.793655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1b6ca-1c03-4877-9e0b-62c584c0d6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:22.813068 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d7e1b6ca-1c03-4877-9e0b-62c584c0d6d5 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:22.814323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d7e1b6ca-1c03-4877-9e0b-62c584c0d6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:22.831788 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:22.874831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1348/2702] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:22.885027 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:22.886819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:22.928206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:22.951301 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a38b5-a853-45b0-bfc9-4dcbbeff93f3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:22.955089 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a332b-d654-4936-9414-28e0f3f41a4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:22.955693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a332b-d654-4936-9414-28e0f3f41a4e service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:22.955967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a332b-d654-4936-9414-28e0f3f41a4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:22.956231 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a332b-d654-4936-9414-28e0f3f41a4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:22.965500 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a332b-d654-4936-9414-28e0f3f41a4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:22.966913 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a332b-d654-4936-9414-28e0f3f41a4e service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:22.975894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1355/2703] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:23.252841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1b6ca-1c03-4877-9e0b-62c584c0d6d5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:23.254416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d7e1b6ca-1c03-4877-9e0b-62c584c0d6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:23.269738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1b6ca-1c03-4877-9e0b-62c584c0d6d5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:23.276153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:23.276860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:23.277122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:23.277445 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] PATH_INFO: `/v3/projects/0827222448714e309da5d8ce8e2292f6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:23.278653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0827222448714e309da5d8ce8e2292f6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:23.280542 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:23.328231 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0827222448714e309da5d8ce8e2292f6', 'request_id': 'req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.336066 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0827222448714e309da5d8ce8e2292f6', 'request_id': 'req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.342398 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0827222448714e309da5d8ce8e2292f6', 'request_id': 'req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.353510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] The token cache is being invalidate because project 0827222448714e309da5d8ce8e2292f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:40:23.353792 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.354409 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.355887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1349/2704] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:40:23 2022] DELETE /identity/v3/projects/0827222448714e309da5d8ce8e2292f6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:23.359988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a332b-d654-4936-9414-28e0f3f41a4e service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:23.369727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a332b-d654-4936-9414-28e0f3f41a4e service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:23.426968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1817d0-c10a-435c-817d-240f63405916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:23.427597 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1817d0-c10a-435c-817d-240f63405916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:23.427871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1817d0-c10a-435c-817d-240f63405916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:23.428142 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1817d0-c10a-435c-817d-240f63405916 admin admin] PATH_INFO: `/v3/users/cf75cb4f56794589ae7af00386bbea0c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:23.429357 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1817d0-c10a-435c-817d-240f63405916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf75cb4f56794589ae7af00386bbea0c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:23.431302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1817d0-c10a-435c-817d-240f63405916 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:23.481397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ef1817d0-c10a-435c-817d-240f63405916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf75cb4f56794589ae7af00386bbea0c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.490024 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ef1817d0-c10a-435c-817d-240f63405916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf75cb4f56794589ae7af00386bbea0c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.490618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ef1817d0-c10a-435c-817d-240f63405916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf75cb4f56794589ae7af00386bbea0c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.497622 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ef1817d0-c10a-435c-817d-240f63405916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf75cb4f56794589ae7af00386bbea0c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.509720 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ef1817d0-c10a-435c-817d-240f63405916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf75cb4f56794589ae7af00386bbea0c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.515948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1356/2705] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:40:23 2022] DELETE /identity/v3/users/cf75cb4f56794589ae7af00386bbea0c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:23.802689 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:23.804298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:23.817708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc3f1e-aeec-400a-8eb1-f14e166e2b95 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:23.877395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d732488-b27a-4d83-ac13-47ce48949d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:23.878025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d732488-b27a-4d83-ac13-47ce48949d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:23.878299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d732488-b27a-4d83-ac13-47ce48949d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:23.878566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d732488-b27a-4d83-ac13-47ce48949d63 admin admin] PATH_INFO: `/v3/projects/cafca35e180d4a43ad474e3e2ea61800` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:23.879680 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d732488-b27a-4d83-ac13-47ce48949d63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cafca35e180d4a43ad474e3e2ea61800)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:23.881633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d732488-b27a-4d83-ac13-47ce48949d63 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:23.928779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:23.930179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7d732488-b27a-4d83-ac13-47ce48949d63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cafca35e180d4a43ad474e3e2ea61800', 'request_id': 'req-7d732488-b27a-4d83-ac13-47ce48949d63'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.938602 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7d732488-b27a-4d83-ac13-47ce48949d63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cafca35e180d4a43ad474e3e2ea61800', 'request_id': 'req-7d732488-b27a-4d83-ac13-47ce48949d63'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.944683 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7d732488-b27a-4d83-ac13-47ce48949d63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cafca35e180d4a43ad474e3e2ea61800', 'request_id': 'req-7d732488-b27a-4d83-ac13-47ce48949d63'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.955865 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7d732488-b27a-4d83-ac13-47ce48949d63 admin admin] The token cache is being invalidate because project cafca35e180d4a43ad474e3e2ea61800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:40:23.956070 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7d732488-b27a-4d83-ac13-47ce48949d63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.956707 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7d732488-b27a-4d83-ac13-47ce48949d63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:23.958257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1350/2706] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:40:23 2022] DELETE /identity/v3/projects/cafca35e180d4a43ad474e3e2ea61800 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:23.964199 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3aa5b-771e-48e3-b58d-ebfe14b4716e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:23.964490 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3aa5b-771e-48e3-b58d-ebfe14b4716e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:23.964762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3aa5b-771e-48e3-b58d-ebfe14b4716e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:23.985269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1de3aa5b-771e-48e3-b58d-ebfe14b4716e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:23.986481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1de3aa5b-771e-48e3-b58d-ebfe14b4716e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:24.036622 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:24.050973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1357/2707] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:24.061588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748f229-3071-4546-8024-11dea52a0010 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.061959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748f229-3071-4546-8024-11dea52a0010 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.062313 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748f229-3071-4546-8024-11dea52a0010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.082454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-3748f229-3071-4546-8024-11dea52a0010 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:24.083663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3748f229-3071-4546-8024-11dea52a0010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:24.092626 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1351/2708] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:24.097015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3aa5b-771e-48e3-b58d-ebfe14b4716e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:24.106936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3aa5b-771e-48e3-b58d-ebfe14b4716e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:24.112410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc8981-dec1-4d60-bfe3-52dd9d9baf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:24.113002 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc8981-dec1-4d60-bfe3-52dd9d9baf95 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.113307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc8981-dec1-4d60-bfe3-52dd9d9baf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.113554 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc8981-dec1-4d60-bfe3-52dd9d9baf95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.114189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc8981-dec1-4d60-bfe3-52dd9d9baf95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:40:24.114487 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc8981-dec1-4d60-bfe3-52dd9d9baf95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:24.116077 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc8981-dec1-4d60-bfe3-52dd9d9baf95 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:24.128737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1358/2709] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:40:24.133177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748f229-3071-4546-8024-11dea52a0010 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:24.142786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748f229-3071-4546-8024-11dea52a0010 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:24.148536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479d498-6aad-4ad9-bd44-97294adf99a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:24.149113 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479d498-6aad-4ad9-bd44-97294adf99a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.149435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479d498-6aad-4ad9-bd44-97294adf99a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.149708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479d498-6aad-4ad9-bd44-97294adf99a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.150334 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479d498-6aad-4ad9-bd44-97294adf99a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:24.152284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479d498-6aad-4ad9-bd44-97294adf99a8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:24.182406 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1352/2710] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:40:24 2022] POST /identity/v3/projects => generated 409 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:40:24.186684 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc8981-dec1-4d60-bfe3-52dd9d9baf95 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:24.196744 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc8981-dec1-4d60-bfe3-52dd9d9baf95 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:24.257553 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bd129-e041-401c-8cfb-7cc4a89f3435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:24.258071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd129-e041-401c-8cfb-7cc4a89f3435 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.258347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd129-e041-401c-8cfb-7cc4a89f3435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.258890 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd129-e041-401c-8cfb-7cc4a89f3435 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.259512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bd129-e041-401c-8cfb-7cc4a89f3435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:24.261497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bd129-e041-401c-8cfb-7cc4a89f3435 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:24.284290 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1359/2711] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:40:24 2022] POST /identity/v3/users => generated 305 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:24.288639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479d498-6aad-4ad9-bd44-97294adf99a8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:24.299243 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479d498-6aad-4ad9-bd44-97294adf99a8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:24.304501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9a822-0895-4b18-9790-e1ce61f48b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:24.305111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9a822-0895-4b18-9790-e1ce61f48b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.305408 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9a822-0895-4b18-9790-e1ce61f48b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.305677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9a822-0895-4b18-9790-e1ce61f48b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.306321 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9a822-0895-4b18-9790-e1ce61f48b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:24.307907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9a822-0895-4b18-9790-e1ce61f48b4c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:24.318203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1353/2712] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:24.322167 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bd129-e041-401c-8cfb-7cc4a89f3435 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:24.332636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bd129-e041-401c-8cfb-7cc4a89f3435 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:24.338813 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a30b96-6ac9-4270-9392-82a970bd337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:24.339418 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a30b96-6ac9-4270-9392-82a970bd337b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.339692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a30b96-6ac9-4270-9392-82a970bd337b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.339948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a30b96-6ac9-4270-9392-82a970bd337b admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.340579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a30b96-6ac9-4270-9392-82a970bd337b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:24.342214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a30b96-6ac9-4270-9392-82a970bd337b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:24.353314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1360/2713] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:24.357732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9a822-0895-4b18-9790-e1ce61f48b4c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:24.368269 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9a822-0895-4b18-9790-e1ce61f48b4c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:24.373961 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02337e41-b298-4905-85a7-888575f297bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:24.374615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02337e41-b298-4905-85a7-888575f297bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.374895 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02337e41-b298-4905-85a7-888575f297bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.375178 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02337e41-b298-4905-85a7-888575f297bf admin admin] PATH_INFO: `/v3/projects/bfc042565fff4334b4c77e8cdfe9177b/users/992103020f964deb81a5fa33014aaa54/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.390170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02337e41-b298-4905-85a7-888575f297bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc042565fff4334b4c77e8cdfe9177b, user_id=992103020f964deb81a5fa33014aaa54, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:24.392799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02337e41-b298-4905-85a7-888575f297bf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:24.406591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1354/2714] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:24 2022] PUT /identity/v3/projects/bfc042565fff4334b4c77e8cdfe9177b/users/992103020f964deb81a5fa33014aaa54/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:24.418087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c37ce8-6a2d-48a2-bb26-35820fddb6fa None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.418627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c37ce8-6a2d-48a2-bb26-35820fddb6fa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.418627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c37ce8-6a2d-48a2-bb26-35820fddb6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.446841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a1c37ce8-6a2d-48a2-bb26-35820fddb6fa None None] MFA Rules not processed for user `992103020f964deb81a5fa33014aaa54`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:24.448054 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a1c37ce8-6a2d-48a2-bb26-35820fddb6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:24.517523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1361/2715] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:24 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:24.522150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02337e41-b298-4905-85a7-888575f297bf admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:24.583867 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02337e41-b298-4905-85a7-888575f297bf admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:24.589842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278390f7-7d48-4c52-bedb-7bdbbcc43544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:24.590452 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278390f7-7d48-4c52-bedb-7bdbbcc43544 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.590724 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278390f7-7d48-4c52-bedb-7bdbbcc43544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.591026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278390f7-7d48-4c52-bedb-7bdbbcc43544 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.591654 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278390f7-7d48-4c52-bedb-7bdbbcc43544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:24.593606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278390f7-7d48-4c52-bedb-7bdbbcc43544 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:24.622377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1355/2716] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:40:24 2022] POST /identity/v3/projects => generated 409 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:24.626650 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c37ce8-6a2d-48a2-bb26-35820fddb6fa None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:24.637906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c37ce8-6a2d-48a2-bb26-35820fddb6fa None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:24.687000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:24.696680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe14c2-2d9c-41ec-972e-2c3a87030711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:24.697326 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe14c2-2d9c-41ec-972e-2c3a87030711 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.697604 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe14c2-2d9c-41ec-972e-2c3a87030711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.697882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe14c2-2d9c-41ec-972e-2c3a87030711 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.698515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe14c2-2d9c-41ec-972e-2c3a87030711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:24.700454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe14c2-2d9c-41ec-972e-2c3a87030711 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:24.723005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1362/2717] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:40:24 2022] POST /identity/v3/users => generated 311 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:24.727174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278390f7-7d48-4c52-bedb-7bdbbcc43544 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:24.737931 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278390f7-7d48-4c52-bedb-7bdbbcc43544 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:24.743487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b323f6-110b-40ca-9d3d-ca976670a887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:24.744084 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b323f6-110b-40ca-9d3d-ca976670a887 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.744360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b323f6-110b-40ca-9d3d-ca976670a887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.744632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b323f6-110b-40ca-9d3d-ca976670a887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.745318 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b323f6-110b-40ca-9d3d-ca976670a887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:24.747083 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b323f6-110b-40ca-9d3d-ca976670a887 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:24.757836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1356/2718] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:24.761968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe14c2-2d9c-41ec-972e-2c3a87030711 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:24.772533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe14c2-2d9c-41ec-972e-2c3a87030711 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:24.778407 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b86eb-1957-49bc-bc90-1189429f8bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:24.779061 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b86eb-1957-49bc-bc90-1189429f8bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.779339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b86eb-1957-49bc-bc90-1189429f8bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.779608 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b86eb-1957-49bc-bc90-1189429f8bf6 admin admin] PATH_INFO: `/v3/projects/465a99f23b334d0a93c069e9ad10f4d7/users/d774153664ce4abe900d2dfb157ae3ab/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.795244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b86eb-1957-49bc-bc90-1189429f8bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=465a99f23b334d0a93c069e9ad10f4d7, user_id=d774153664ce4abe900d2dfb157ae3ab, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:24.797951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b86eb-1957-49bc-bc90-1189429f8bf6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:24.811963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1363/2719] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:24 2022] PUT /identity/v3/projects/465a99f23b334d0a93c069e9ad10f4d7/users/d774153664ce4abe900d2dfb157ae3ab/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:40:24.815966 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b323f6-110b-40ca-9d3d-ca976670a887 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:24.826235 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b323f6-110b-40ca-9d3d-ca976670a887 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:24.870967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:24.885007 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24a499-645e-48ef-87e6-e86a2b8b72a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:24.885729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24a499-645e-48ef-87e6-e86a2b8b72a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.886010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24a499-645e-48ef-87e6-e86a2b8b72a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.886282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24a499-645e-48ef-87e6-e86a2b8b72a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.886952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24a499-645e-48ef-87e6-e86a2b8b72a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:24.888566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24a499-645e-48ef-87e6-e86a2b8b72a9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:24.898186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1357/2720] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:24.902766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b86eb-1957-49bc-bc90-1189429f8bf6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:24.913456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b86eb-1957-49bc-bc90-1189429f8bf6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:24.919550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47380dbb-42e7-4e4e-b002-293ff366e0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:24.920236 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47380dbb-42e7-4e4e-b002-293ff366e0b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.920778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47380dbb-42e7-4e4e-b002-293ff366e0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.921038 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47380dbb-42e7-4e4e-b002-293ff366e0b2 admin admin] PATH_INFO: `/v3/projects/465a99f23b334d0a93c069e9ad10f4d7/users/d774153664ce4abe900d2dfb157ae3ab/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.923226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47380dbb-42e7-4e4e-b002-293ff366e0b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=465a99f23b334d0a93c069e9ad10f4d7, user_id=d774153664ce4abe900d2dfb157ae3ab, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:24.926141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47380dbb-42e7-4e4e-b002-293ff366e0b2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:24.937108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1364/2721] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:24 2022] PUT /identity/v3/projects/465a99f23b334d0a93c069e9ad10f4d7/users/d774153664ce4abe900d2dfb157ae3ab/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:40:24.947363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738767d-2d7e-4802-bb96-d7c777979ecd None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:24.947648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738767d-2d7e-4802-bb96-d7c777979ecd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:24.947917 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738767d-2d7e-4802-bb96-d7c777979ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:24.982958 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c738767d-2d7e-4802-bb96-d7c777979ecd None None] MFA Rules not processed for user `d774153664ce4abe900d2dfb157ae3ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:24.984174 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c738767d-2d7e-4802-bb96-d7c777979ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:25.054100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1358/2722] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:24 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:25.065157 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47380dbb-42e7-4e4e-b002-293ff366e0b2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:25.067312 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-47380dbb-42e7-4e4e-b002-293ff366e0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:25.131051 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47380dbb-42e7-4e4e-b002-293ff366e0b2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:25.136700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57500d05-7209-488b-be03-bb9417351176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:25.137534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57500d05-7209-488b-be03-bb9417351176 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:25.137806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57500d05-7209-488b-be03-bb9417351176 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:25.138073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57500d05-7209-488b-be03-bb9417351176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:25.150654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57500d05-7209-488b-be03-bb9417351176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:25.152130 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57500d05-7209-488b-be03-bb9417351176 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:25.165824 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1365/2723] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:25 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:25.328264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738767d-2d7e-4802-bb96-d7c777979ecd None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:25.340173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738767d-2d7e-4802-bb96-d7c777979ecd None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:25.344853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45c441-5083-4721-9a44-85d456dd8068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:25.345492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45c441-5083-4721-9a44-85d456dd8068 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:25.345766 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45c441-5083-4721-9a44-85d456dd8068 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:25.346033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45c441-5083-4721-9a44-85d456dd8068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:25.404936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45c441-5083-4721-9a44-85d456dd8068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:25.406416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45c441-5083-4721-9a44-85d456dd8068 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:25.415354 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1359/2724] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:25 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:34.964511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57500d05-7209-488b-be03-bb9417351176 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:35.027193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57500d05-7209-488b-be03-bb9417351176 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:35.032115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:35.032738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:35.033014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:35.033319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] PATH_INFO: `/v3/users/992103020f964deb81a5fa33014aaa54` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:35.034524 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] RBAC: Authorizing `identity:delete_user(user_id=992103020f964deb81a5fa33014aaa54)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:35.036429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:35.083234 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '992103020f964deb81a5fa33014aaa54'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.085652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:35.091119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '992103020f964deb81a5fa33014aaa54'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.091961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '992103020f964deb81a5fa33014aaa54'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.099861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '992103020f964deb81a5fa33014aaa54'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.111854 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '992103020f964deb81a5fa33014aaa54'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.118262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1366/2725] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:40:34 2022] DELETE /identity/v3/users/992103020f964deb81a5fa33014aaa54 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:35.123015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec712f27-37ac-4836-947d-eea8251c5593 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:35.123515 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec712f27-37ac-4836-947d-eea8251c5593 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:35.123627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec712f27-37ac-4836-947d-eea8251c5593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:35.143779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ec712f27-37ac-4836-947d-eea8251c5593 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:35.144992 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ec712f27-37ac-4836-947d-eea8251c5593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:35.206161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1360/2726] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:35.216542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:35.218083 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:35.279006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7347c90-87c2-4c6a-b51e-2c71bac9e30e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:35.283515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d227f9-cd7a-466c-8ec2-8a30471feb22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:35.284147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d227f9-cd7a-466c-8ec2-8a30471feb22 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:35.284423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d227f9-cd7a-466c-8ec2-8a30471feb22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:35.284687 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d227f9-cd7a-466c-8ec2-8a30471feb22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:35.294535 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d227f9-cd7a-466c-8ec2-8a30471feb22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:35.296201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d227f9-cd7a-466c-8ec2-8a30471feb22 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:35.305941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1367/2727] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:35.586001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec712f27-37ac-4836-947d-eea8251c5593 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:35.587526 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ec712f27-37ac-4836-947d-eea8251c5593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:35.600340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec712f27-37ac-4836-947d-eea8251c5593 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:35.605042 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:35.605704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:35.606002 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:35.606270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] PATH_INFO: `/v3/projects/bfc042565fff4334b4c77e8cdfe9177b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:35.607446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfc042565fff4334b4c77e8cdfe9177b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:35.609491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:35.611144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:35.659040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfc042565fff4334b4c77e8cdfe9177b', 'request_id': 'req-a973daff-85db-4053-81e0-e9e78fbbfd11'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.669779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfc042565fff4334b4c77e8cdfe9177b', 'request_id': 'req-a973daff-85db-4053-81e0-e9e78fbbfd11'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.678190 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfc042565fff4334b4c77e8cdfe9177b', 'request_id': 'req-a973daff-85db-4053-81e0-e9e78fbbfd11'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.695878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] The token cache is being invalidate because project bfc042565fff4334b4c77e8cdfe9177b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:40:35.696305 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.697513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.700097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1361/2728] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:40:35 2022] DELETE /identity/v3/projects/bfc042565fff4334b4c77e8cdfe9177b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:35.704649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d227f9-cd7a-466c-8ec2-8a30471feb22 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:35.714771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d227f9-cd7a-466c-8ec2-8a30471feb22 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:35.779618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995acac6-b209-4b66-9722-96697b3326d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:35.780546 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995acac6-b209-4b66-9722-96697b3326d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:35.780897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995acac6-b209-4b66-9722-96697b3326d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:35.781516 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995acac6-b209-4b66-9722-96697b3326d9 admin admin] PATH_INFO: `/v3/users/d774153664ce4abe900d2dfb157ae3ab` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:35.782751 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995acac6-b209-4b66-9722-96697b3326d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d774153664ce4abe900d2dfb157ae3ab)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:35.784683 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995acac6-b209-4b66-9722-96697b3326d9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:35.833397 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-995acac6-b209-4b66-9722-96697b3326d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd774153664ce4abe900d2dfb157ae3ab'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.842396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-995acac6-b209-4b66-9722-96697b3326d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd774153664ce4abe900d2dfb157ae3ab'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.843114 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-995acac6-b209-4b66-9722-96697b3326d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd774153664ce4abe900d2dfb157ae3ab'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.850648 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-995acac6-b209-4b66-9722-96697b3326d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd774153664ce4abe900d2dfb157ae3ab'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.857663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:35.863928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-995acac6-b209-4b66-9722-96697b3326d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd774153664ce4abe900d2dfb157ae3ab'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:35.870726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1368/2729] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:40:35 2022] DELETE /identity/v3/users/d774153664ce4abe900d2dfb157ae3ab => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:36.154510 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:36.156166 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:36.169558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973daff-85db-4053-81e0-e9e78fbbfd11 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:36.228164 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:36.228715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:36.228990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:36.229295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9 admin admin] PATH_INFO: `/v3/projects/465a99f23b334d0a93c069e9ad10f4d7` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:36.230440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=465a99f23b334d0a93c069e9ad10f4d7)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:36.232769 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:36.284157 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465a99f23b334d0a93c069e9ad10f4d7', 'request_id': 'req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:36.291199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465a99f23b334d0a93c069e9ad10f4d7', 'request_id': 'req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:36.296883 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465a99f23b334d0a93c069e9ad10f4d7', 'request_id': 'req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:36.307953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9 admin admin] The token cache is being invalidate because project 465a99f23b334d0a93c069e9ad10f4d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:40:36.308233 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:36.308939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4a5aef51-ab99-4c23-bcb5-da94b1efaeb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:36.310542 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1362/2730] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:40:36 2022] DELETE /identity/v3/projects/465a99f23b334d0a93c069e9ad10f4d7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:36.315811 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaacb78-ec2e-4b4c-abc7-1906c1cb4227 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:36.316285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaacb78-ec2e-4b4c-abc7-1906c1cb4227 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:36.316409 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaacb78-ec2e-4b4c-abc7-1906c1cb4227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:36.336512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-edaacb78-ec2e-4b4c-abc7-1906c1cb4227 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:36.337750 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-edaacb78-ec2e-4b4c-abc7-1906c1cb4227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:36.408165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1369/2731] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:36.416517 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fb6f4-4efe-4308-9574-0f0b2e0ba3e1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:36.416802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fb6f4-4efe-4308-9574-0f0b2e0ba3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:36.417108 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fb6f4-4efe-4308-9574-0f0b2e0ba3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:36.437349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e46fb6f4-4efe-4308-9574-0f0b2e0ba3e1 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:36.438598 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e46fb6f4-4efe-4308-9574-0f0b2e0ba3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:36.446859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1363/2732] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:36.451730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaacb78-ec2e-4b4c-abc7-1906c1cb4227 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:36.466403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaacb78-ec2e-4b4c-abc7-1906c1cb4227 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:36.472718 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a152c8-b2c0-409c-8943-3279cac96feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:36.473346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a152c8-b2c0-409c-8943-3279cac96feb admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:36.473619 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a152c8-b2c0-409c-8943-3279cac96feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:36.473889 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a152c8-b2c0-409c-8943-3279cac96feb admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:36.474534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a152c8-b2c0-409c-8943-3279cac96feb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:40:36.474830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a152c8-b2c0-409c-8943-3279cac96feb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:36.476446 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a152c8-b2c0-409c-8943-3279cac96feb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:36.488748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1370/2733] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:40:36.493073 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fb6f4-4efe-4308-9574-0f0b2e0ba3e1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:36.502663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fb6f4-4efe-4308-9574-0f0b2e0ba3e1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:36.508614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbfa11-2bf9-48d7-a99b-4388b9f576fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:36.509279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbfa11-2bf9-48d7-a99b-4388b9f576fb admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:36.509563 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbfa11-2bf9-48d7-a99b-4388b9f576fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:36.509825 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbfa11-2bf9-48d7-a99b-4388b9f576fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:36.510448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbfa11-2bf9-48d7-a99b-4388b9f576fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:36.512335 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbfa11-2bf9-48d7-a99b-4388b9f576fb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:36.524251 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:36.545203 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1364/2734] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:40:36 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:36.546458 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a152c8-b2c0-409c-8943-3279cac96feb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:36.556532 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a152c8-b2c0-409c-8943-3279cac96feb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:36.613805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85663d-635d-4f11-9d9e-2938c2d3d535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:36.614395 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85663d-635d-4f11-9d9e-2938c2d3d535 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:36.614665 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85663d-635d-4f11-9d9e-2938c2d3d535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:36.614927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85663d-635d-4f11-9d9e-2938c2d3d535 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:36.615657 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85663d-635d-4f11-9d9e-2938c2d3d535 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:36.617581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85663d-635d-4f11-9d9e-2938c2d3d535 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:36.640174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1371/2735] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:40:36 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:36.644491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbfa11-2bf9-48d7-a99b-4388b9f576fb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:36.655005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbfa11-2bf9-48d7-a99b-4388b9f576fb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:36.660742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cff15-6f6e-465b-b5fe-29b0a7f88588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:36.661402 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cff15-6f6e-465b-b5fe-29b0a7f88588 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:36.661675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cff15-6f6e-465b-b5fe-29b0a7f88588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:36.661918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cff15-6f6e-465b-b5fe-29b0a7f88588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:36.662576 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cff15-6f6e-465b-b5fe-29b0a7f88588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:36.664155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cff15-6f6e-465b-b5fe-29b0a7f88588 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:36.675115 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1365/2736] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:36.679471 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85663d-635d-4f11-9d9e-2938c2d3d535 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:36.691598 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85663d-635d-4f11-9d9e-2938c2d3d535 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:36.696574 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217e817-e00c-4776-ba31-b832e62b89d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:36.697217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217e817-e00c-4776-ba31-b832e62b89d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:36.697534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217e817-e00c-4776-ba31-b832e62b89d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:36.697806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217e817-e00c-4776-ba31-b832e62b89d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:36.698479 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217e817-e00c-4776-ba31-b832e62b89d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:36.700076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217e817-e00c-4776-ba31-b832e62b89d2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:36.710818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1372/2737] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:36.715388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cff15-6f6e-465b-b5fe-29b0a7f88588 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:36.725262 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cff15-6f6e-465b-b5fe-29b0a7f88588 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:36.730414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9bb34-8a05-4c69-b824-43f72ad6405b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:36.731050 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9bb34-8a05-4c69-b824-43f72ad6405b admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:36.731321 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9bb34-8a05-4c69-b824-43f72ad6405b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:36.731612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9bb34-8a05-4c69-b824-43f72ad6405b admin admin] PATH_INFO: `/v3/projects/c9473f4ee35249589c394389562ceb1a/users/e00770c48917427386bd82ab23f3b7da/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:36.746297 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9bb34-8a05-4c69-b824-43f72ad6405b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9473f4ee35249589c394389562ceb1a, user_id=e00770c48917427386bd82ab23f3b7da, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:36.748846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9bb34-8a05-4c69-b824-43f72ad6405b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:36.763156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1366/2738] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:36 2022] PUT /identity/v3/projects/c9473f4ee35249589c394389562ceb1a/users/e00770c48917427386bd82ab23f3b7da/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:36.833075 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82db1b-92ab-4281-a253-ed90a33a8be3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:36.833464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82db1b-92ab-4281-a253-ed90a33a8be3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:36.833927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82db1b-92ab-4281-a253-ed90a33a8be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:36.872131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3f82db1b-92ab-4281-a253-ed90a33a8be3 None None] MFA Rules not processed for user `e00770c48917427386bd82ab23f3b7da`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:36.873408 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3f82db1b-92ab-4281-a253-ed90a33a8be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:36.918842 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:36.943163 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1373/2739] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:36.947835 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9bb34-8a05-4c69-b824-43f72ad6405b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:37.011095 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9bb34-8a05-4c69-b824-43f72ad6405b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:37.016228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c8cbb-cdd0-4b75-82f2-40a77912ec53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:37.016820 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c8cbb-cdd0-4b75-82f2-40a77912ec53 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:37.017054 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c8cbb-cdd0-4b75-82f2-40a77912ec53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:37.017384 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c8cbb-cdd0-4b75-82f2-40a77912ec53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:37.017994 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c8cbb-cdd0-4b75-82f2-40a77912ec53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:37.019862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c8cbb-cdd0-4b75-82f2-40a77912ec53 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:37.048169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1367/2740] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:40:36 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:37.052325 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82db1b-92ab-4281-a253-ed90a33a8be3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:37.063577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82db1b-92ab-4281-a253-ed90a33a8be3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:37.121943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90a493-5b56-4e94-8084-f55b07009d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:37.122544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90a493-5b56-4e94-8084-f55b07009d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:37.122821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90a493-5b56-4e94-8084-f55b07009d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:37.123091 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90a493-5b56-4e94-8084-f55b07009d8b admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:37.123715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90a493-5b56-4e94-8084-f55b07009d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:37.125696 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90a493-5b56-4e94-8084-f55b07009d8b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:37.148934 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1374/2741] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:40:37 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:37.153155 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c8cbb-cdd0-4b75-82f2-40a77912ec53 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:37.165670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c8cbb-cdd0-4b75-82f2-40a77912ec53 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:37.172504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5660923-2339-4689-8ed8-f1eb24c72ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:37.173117 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5660923-2339-4689-8ed8-f1eb24c72ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:37.173399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5660923-2339-4689-8ed8-f1eb24c72ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:37.173667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5660923-2339-4689-8ed8-f1eb24c72ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:37.174313 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5660923-2339-4689-8ed8-f1eb24c72ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:37.175895 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5660923-2339-4689-8ed8-f1eb24c72ea0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:37.186600 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1368/2742] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:37.191319 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90a493-5b56-4e94-8084-f55b07009d8b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:37.201565 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90a493-5b56-4e94-8084-f55b07009d8b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:37.206978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66809c-a3b5-47c0-9894-add1f2b2f640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:37.207647 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66809c-a3b5-47c0-9894-add1f2b2f640 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:37.207918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66809c-a3b5-47c0-9894-add1f2b2f640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:37.208188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66809c-a3b5-47c0-9894-add1f2b2f640 admin admin] PATH_INFO: `/v3/projects/87c4417185834e95b51a6181636093a3/users/3d5627aefceb4135be6de5a778b46ff3/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:37.223566 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66809c-a3b5-47c0-9894-add1f2b2f640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c4417185834e95b51a6181636093a3, user_id=3d5627aefceb4135be6de5a778b46ff3, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:37.226229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66809c-a3b5-47c0-9894-add1f2b2f640 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:37.240921 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1375/2743] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:37 2022] PUT /identity/v3/projects/87c4417185834e95b51a6181636093a3/users/3d5627aefceb4135be6de5a778b46ff3/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:37.246272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5660923-2339-4689-8ed8-f1eb24c72ea0 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:37.256706 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5660923-2339-4689-8ed8-f1eb24c72ea0 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:37.313993 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4901f-8f7c-4a17-8760-4b8d689fa7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:37.314591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4901f-8f7c-4a17-8760-4b8d689fa7da admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:37.314868 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4901f-8f7c-4a17-8760-4b8d689fa7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:37.315138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4901f-8f7c-4a17-8760-4b8d689fa7da admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:37.315775 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4901f-8f7c-4a17-8760-4b8d689fa7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:37.317404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4901f-8f7c-4a17-8760-4b8d689fa7da admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:37.327113 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1369/2744] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:37.331563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66809c-a3b5-47c0-9894-add1f2b2f640 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:37.342284 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66809c-a3b5-47c0-9894-add1f2b2f640 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:37.347808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7f453-c6cb-4cd1-815d-6ac5540952a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:37.348506 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7f453-c6cb-4cd1-815d-6ac5540952a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:37.348821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7f453-c6cb-4cd1-815d-6ac5540952a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:37.349139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7f453-c6cb-4cd1-815d-6ac5540952a2 admin admin] PATH_INFO: `/v3/projects/87c4417185834e95b51a6181636093a3/users/3d5627aefceb4135be6de5a778b46ff3/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:37.351223 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7f453-c6cb-4cd1-815d-6ac5540952a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c4417185834e95b51a6181636093a3, user_id=3d5627aefceb4135be6de5a778b46ff3, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:37.353881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7f453-c6cb-4cd1-815d-6ac5540952a2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:37.364602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1376/2745] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:37 2022] PUT /identity/v3/projects/87c4417185834e95b51a6181636093a3/users/3d5627aefceb4135be6de5a778b46ff3/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:37.375032 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb515876-b5e0-4990-95d4-01c945e6eb09 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:37.375323 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb515876-b5e0-4990-95d4-01c945e6eb09 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:37.375967 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb515876-b5e0-4990-95d4-01c945e6eb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:37.407057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fb515876-b5e0-4990-95d4-01c945e6eb09 None None] MFA Rules not processed for user `3d5627aefceb4135be6de5a778b46ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:37.408325 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fb515876-b5e0-4990-95d4-01c945e6eb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:37.428859 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:37.491583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1370/2746] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:40:37.503384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7f453-c6cb-4cd1-815d-6ac5540952a2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:37.505428 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7db7f453-c6cb-4cd1-815d-6ac5540952a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:37.569200 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7f453-c6cb-4cd1-815d-6ac5540952a2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:37.572809 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790b195-3dc5-4764-a91f-86c1d7b6e5df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:37.573462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790b195-3dc5-4764-a91f-86c1d7b6e5df service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:37.573716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790b195-3dc5-4764-a91f-86c1d7b6e5df service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:37.573975 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790b195-3dc5-4764-a91f-86c1d7b6e5df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:37.583948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790b195-3dc5-4764-a91f-86c1d7b6e5df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:37.585404 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790b195-3dc5-4764-a91f-86c1d7b6e5df service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:37.597193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1377/2747] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:37.677048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb515876-b5e0-4990-95d4-01c945e6eb09 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:37.688889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb515876-b5e0-4990-95d4-01c945e6eb09 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:37.692560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59286946-ba12-4447-bf98-17053f2b97d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:37.693187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286946-ba12-4447-bf98-17053f2b97d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:37.693464 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286946-ba12-4447-bf98-17053f2b97d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:37.693729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286946-ba12-4447-bf98-17053f2b97d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:37.756005 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59286946-ba12-4447-bf98-17053f2b97d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:37.757485 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59286946-ba12-4447-bf98-17053f2b97d4 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:37.766708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1371/2748] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:38.042440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790b195-3dc5-4764-a91f-86c1d7b6e5df service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:38.084347 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:38.105394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790b195-3dc5-4764-a91f-86c1d7b6e5df service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:38.110456 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:38.111087 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:38.111366 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:38.111630 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] PATH_INFO: `/v3/users/e00770c48917427386bd82ab23f3b7da` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:38.112816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e00770c48917427386bd82ab23f3b7da)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:38.114773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:38.161385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e00770c48917427386bd82ab23f3b7da'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.169550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e00770c48917427386bd82ab23f3b7da'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.170209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e00770c48917427386bd82ab23f3b7da'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.177289 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e00770c48917427386bd82ab23f3b7da'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.190288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e00770c48917427386bd82ab23f3b7da'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.196196 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1378/2749] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:40:38 2022] DELETE /identity/v3/users/e00770c48917427386bd82ab23f3b7da => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:38.200953 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff7595-443f-4037-8b5a-c41262cc6f57 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:38.201481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff7595-443f-4037-8b5a-c41262cc6f57 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:38.201481 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff7595-443f-4037-8b5a-c41262cc6f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:38.221729 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-09ff7595-443f-4037-8b5a-c41262cc6f57 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:38.222974 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-09ff7595-443f-4037-8b5a-c41262cc6f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:38.289747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1372/2750] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:38.301603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:38.303215 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:38.365031 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb202d-7c89-4c25-b3ee-d8b21080e85a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:38.369177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d354d16-6131-4ed8-bdbb-7df9c8c16f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:38.369784 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d354d16-6131-4ed8-bdbb-7df9c8c16f0d service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:38.370057 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d354d16-6131-4ed8-bdbb-7df9c8c16f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:38.370329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d354d16-6131-4ed8-bdbb-7df9c8c16f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:38.380208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d354d16-6131-4ed8-bdbb-7df9c8c16f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:38.381681 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d354d16-6131-4ed8-bdbb-7df9c8c16f0d service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:38.391111 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1379/2751] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:38.659095 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff7595-443f-4037-8b5a-c41262cc6f57 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:38.660538 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-09ff7595-443f-4037-8b5a-c41262cc6f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:38.673968 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff7595-443f-4037-8b5a-c41262cc6f57 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:38.679392 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:38.680033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:38.680307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:38.680571 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] PATH_INFO: `/v3/projects/c9473f4ee35249589c394389562ceb1a` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:38.681853 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9473f4ee35249589c394389562ceb1a)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:38.683615 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:38.737921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:38.741228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9473f4ee35249589c394389562ceb1a', 'request_id': 'req-b7743781-9ba7-41fe-9cbb-376536fa45b9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.749401 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9473f4ee35249589c394389562ceb1a', 'request_id': 'req-b7743781-9ba7-41fe-9cbb-376536fa45b9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.755126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9473f4ee35249589c394389562ceb1a', 'request_id': 'req-b7743781-9ba7-41fe-9cbb-376536fa45b9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.766223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] The token cache is being invalidate because project c9473f4ee35249589c394389562ceb1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:40:38.766504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.767122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.768578 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1373/2752] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:40:38 2022] DELETE /identity/v3/projects/c9473f4ee35249589c394389562ceb1a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:38.772748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d354d16-6131-4ed8-bdbb-7df9c8c16f0d service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:38.782464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d354d16-6131-4ed8-bdbb-7df9c8c16f0d service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:38.839281 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167749f-81b5-486b-968e-faadd702504e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:38.839905 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167749f-81b5-486b-968e-faadd702504e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:38.840188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167749f-81b5-486b-968e-faadd702504e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:38.840439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167749f-81b5-486b-968e-faadd702504e admin admin] PATH_INFO: `/v3/users/3d5627aefceb4135be6de5a778b46ff3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:38.841612 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167749f-81b5-486b-968e-faadd702504e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d5627aefceb4135be6de5a778b46ff3)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:38.843540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167749f-81b5-486b-968e-faadd702504e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:38.845927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:38.891819 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9167749f-81b5-486b-968e-faadd702504e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d5627aefceb4135be6de5a778b46ff3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.900117 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9167749f-81b5-486b-968e-faadd702504e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d5627aefceb4135be6de5a778b46ff3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.901003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9167749f-81b5-486b-968e-faadd702504e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d5627aefceb4135be6de5a778b46ff3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.910640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9167749f-81b5-486b-968e-faadd702504e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d5627aefceb4135be6de5a778b46ff3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.924096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9167749f-81b5-486b-968e-faadd702504e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d5627aefceb4135be6de5a778b46ff3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:38.930259 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1380/2753] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:40:38 2022] DELETE /identity/v3/users/3d5627aefceb4135be6de5a778b46ff3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:39.211555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:39.213439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:39.226270 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7743781-9ba7-41fe-9cbb-376536fa45b9 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:39.284412 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914080a-25a7-4df2-86f0-cc2ae82e07f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:39.285049 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914080a-25a7-4df2-86f0-cc2ae82e07f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:39.285368 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914080a-25a7-4df2-86f0-cc2ae82e07f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:39.285636 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914080a-25a7-4df2-86f0-cc2ae82e07f9 admin admin] PATH_INFO: `/v3/projects/87c4417185834e95b51a6181636093a3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:39.286797 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914080a-25a7-4df2-86f0-cc2ae82e07f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87c4417185834e95b51a6181636093a3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:39.288694 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914080a-25a7-4df2-86f0-cc2ae82e07f9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:39.337280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8914080a-25a7-4df2-86f0-cc2ae82e07f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87c4417185834e95b51a6181636093a3', 'request_id': 'req-8914080a-25a7-4df2-86f0-cc2ae82e07f9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:39.345773 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8914080a-25a7-4df2-86f0-cc2ae82e07f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87c4417185834e95b51a6181636093a3', 'request_id': 'req-8914080a-25a7-4df2-86f0-cc2ae82e07f9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:39.351550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8914080a-25a7-4df2-86f0-cc2ae82e07f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87c4417185834e95b51a6181636093a3', 'request_id': 'req-8914080a-25a7-4df2-86f0-cc2ae82e07f9'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:39.362298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8914080a-25a7-4df2-86f0-cc2ae82e07f9 admin admin] The token cache is being invalidate because project 87c4417185834e95b51a6181636093a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:40:39.362590 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8914080a-25a7-4df2-86f0-cc2ae82e07f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:39.363232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8914080a-25a7-4df2-86f0-cc2ae82e07f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:39.364714 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1374/2754] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:40:39 2022] DELETE /identity/v3/projects/87c4417185834e95b51a6181636093a3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:39.370190 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628c423-1ab0-4637-a816-c8fc4dce822d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:39.370680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628c423-1ab0-4637-a816-c8fc4dce822d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:39.370796 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628c423-1ab0-4637-a816-c8fc4dce822d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:39.390655 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a628c423-1ab0-4637-a816-c8fc4dce822d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:39.391896 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a628c423-1ab0-4637-a816-c8fc4dce822d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:39.452165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1381/2755] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:39.461308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ddee4-0b08-4a2f-b803-df9b419c6483 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:39.461467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ddee4-0b08-4a2f-b803-df9b419c6483 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:39.461774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ddee4-0b08-4a2f-b803-df9b419c6483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:39.483627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b98ddee4-0b08-4a2f-b803-df9b419c6483 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:39.484816 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b98ddee4-0b08-4a2f-b803-df9b419c6483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:39.493410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1375/2756] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:39.497872 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628c423-1ab0-4637-a816-c8fc4dce822d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:39.507959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628c423-1ab0-4637-a816-c8fc4dce822d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:39.513498 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbd8ca-a9bc-4c52-896e-63937671f919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:39.514127 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbd8ca-a9bc-4c52-896e-63937671f919 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:39.514401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbd8ca-a9bc-4c52-896e-63937671f919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:39.514666 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbd8ca-a9bc-4c52-896e-63937671f919 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:39.515363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbd8ca-a9bc-4c52-896e-63937671f919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:40:39.515661 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbd8ca-a9bc-4c52-896e-63937671f919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:39.517318 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbd8ca-a9bc-4c52-896e-63937671f919 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:39.529298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1382/2757] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:40:39.533618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ddee4-0b08-4a2f-b803-df9b419c6483 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:39.542933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ddee4-0b08-4a2f-b803-df9b419c6483 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:39.548337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354284e3-3b11-4719-bc53-906fad5db3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:39.548972 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354284e3-3b11-4719-bc53-906fad5db3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:39.549293 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354284e3-3b11-4719-bc53-906fad5db3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:39.549605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354284e3-3b11-4719-bc53-906fad5db3b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:39.550230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354284e3-3b11-4719-bc53-906fad5db3b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:39.552133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354284e3-3b11-4719-bc53-906fad5db3b3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:39.588995 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1376/2758] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:40:39 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:39.593173 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbd8ca-a9bc-4c52-896e-63937671f919 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:39.603007 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbd8ca-a9bc-4c52-896e-63937671f919 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:39.663717 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374914c-d6c4-41b0-bcf2-af8cc98f316b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:39.664348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374914c-d6c4-41b0-bcf2-af8cc98f316b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:39.664618 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374914c-d6c4-41b0-bcf2-af8cc98f316b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:39.664886 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374914c-d6c4-41b0-bcf2-af8cc98f316b admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:39.665570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374914c-d6c4-41b0-bcf2-af8cc98f316b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:39.667464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374914c-d6c4-41b0-bcf2-af8cc98f316b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:39.685576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:39.692985 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1383/2759] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:40:39 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:39.696455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354284e3-3b11-4719-bc53-906fad5db3b3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:39.707098 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354284e3-3b11-4719-bc53-906fad5db3b3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:39.712650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541b566-c11f-4409-a424-66dd131e1a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:39.713587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541b566-c11f-4409-a424-66dd131e1a45 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:39.713869 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541b566-c11f-4409-a424-66dd131e1a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:39.714144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541b566-c11f-4409-a424-66dd131e1a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:39.714794 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541b566-c11f-4409-a424-66dd131e1a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:39.716396 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541b566-c11f-4409-a424-66dd131e1a45 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:39.726807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1377/2760] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:39.730968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374914c-d6c4-41b0-bcf2-af8cc98f316b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:39.741415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374914c-d6c4-41b0-bcf2-af8cc98f316b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:39.746980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7ddd2-90cc-4371-862c-83c3a3b070b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:39.747563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7ddd2-90cc-4371-862c-83c3a3b070b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:39.747861 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7ddd2-90cc-4371-862c-83c3a3b070b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:39.748126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7ddd2-90cc-4371-862c-83c3a3b070b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:39.748768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7ddd2-90cc-4371-862c-83c3a3b070b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:39.750370 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7ddd2-90cc-4371-862c-83c3a3b070b2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:39.761234 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1384/2761] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:39.765427 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541b566-c11f-4409-a424-66dd131e1a45 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:39.775984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541b566-c11f-4409-a424-66dd131e1a45 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:39.781573 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875ef8a-b1ce-4ce9-b71d-c1404c1608ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:39.782218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875ef8a-b1ce-4ce9-b71d-c1404c1608ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:39.782503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875ef8a-b1ce-4ce9-b71d-c1404c1608ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:39.782772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875ef8a-b1ce-4ce9-b71d-c1404c1608ac admin admin] PATH_INFO: `/v3/projects/d75c56ab86ca415dabb82bacc7d25e6d/users/3066e7957a364658b54551b691005c93/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:39.800080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875ef8a-b1ce-4ce9-b71d-c1404c1608ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75c56ab86ca415dabb82bacc7d25e6d, user_id=3066e7957a364658b54551b691005c93, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:39.804074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875ef8a-b1ce-4ce9-b71d-c1404c1608ac admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:39.816723 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:39.822165 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1378/2762] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:39 2022] PUT /identity/v3/projects/d75c56ab86ca415dabb82bacc7d25e6d/users/3066e7957a364658b54551b691005c93/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:39.832690 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea6fdd-0653-4d82-b32d-0b0168f2ea95 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:39.832968 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea6fdd-0653-4d82-b32d-0b0168f2ea95 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:39.833265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea6fdd-0653-4d82-b32d-0b0168f2ea95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:39.861262 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b2ea6fdd-0653-4d82-b32d-0b0168f2ea95 None None] MFA Rules not processed for user `3066e7957a364658b54551b691005c93`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:39.862452 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b2ea6fdd-0653-4d82-b32d-0b0168f2ea95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:39.926385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1385/2763] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:39.937671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875ef8a-b1ce-4ce9-b71d-c1404c1608ac admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:39.939950 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8875ef8a-b1ce-4ce9-b71d-c1404c1608ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:39.999667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875ef8a-b1ce-4ce9-b71d-c1404c1608ac admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:40.003730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2502e04-f5cf-4d7f-9704-8823ef5630d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:40.004333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2502e04-f5cf-4d7f-9704-8823ef5630d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:40.004612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2502e04-f5cf-4d7f-9704-8823ef5630d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:40.004889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2502e04-f5cf-4d7f-9704-8823ef5630d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:40.013848 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2502e04-f5cf-4d7f-9704-8823ef5630d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:40.015276 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2502e04-f5cf-4d7f-9704-8823ef5630d6 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:40.025200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1379/2764] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:49.564821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea6fdd-0653-4d82-b32d-0b0168f2ea95 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:49.627940 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea6fdd-0653-4d82-b32d-0b0168f2ea95 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:49.632916 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:49.633578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:49.633814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:49.634130 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] PATH_INFO: `/v3/users/3066e7957a364658b54551b691005c93` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:49.635249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3066e7957a364658b54551b691005c93)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:49.637168 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:49.687392 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3066e7957a364658b54551b691005c93'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:49.695160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3066e7957a364658b54551b691005c93'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:49.695818 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3066e7957a364658b54551b691005c93'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:49.703546 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3066e7957a364658b54551b691005c93'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:49.715775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:49.717027 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3066e7957a364658b54551b691005c93'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:49.724555 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1386/2765] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:40:49 2022] DELETE /identity/v3/users/3066e7957a364658b54551b691005c93 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:49.729076 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634df366-eacd-4271-a05f-bbf726d8b81e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:49.729558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634df366-eacd-4271-a05f-bbf726d8b81e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:49.729664 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634df366-eacd-4271-a05f-bbf726d8b81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:49.748391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-634df366-eacd-4271-a05f-bbf726d8b81e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:49.749601 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-634df366-eacd-4271-a05f-bbf726d8b81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:49.808019 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1380/2766] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:49.817874 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:49.819372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:49.876099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c104aa0-2808-40e8-bb6a-042b9e0f1287 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:49.880160 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975a49c-ec9a-4512-93a1-a14fef80235f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:49.880840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a49c-ec9a-4512-93a1-a14fef80235f service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:49.881130 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a49c-ec9a-4512-93a1-a14fef80235f service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:49.881399 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a49c-ec9a-4512-93a1-a14fef80235f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:49.890378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975a49c-ec9a-4512-93a1-a14fef80235f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:49.891783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975a49c-ec9a-4512-93a1-a14fef80235f service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:49.900951 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1387/2767] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:40:50.171513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634df366-eacd-4271-a05f-bbf726d8b81e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:50.173024 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-634df366-eacd-4271-a05f-bbf726d8b81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:50.186341 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634df366-eacd-4271-a05f-bbf726d8b81e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:50.191460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:50.192109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:50.192387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:50.192693 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b admin admin] PATH_INFO: `/v3/projects/d75c56ab86ca415dabb82bacc7d25e6d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:50.193933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75c56ab86ca415dabb82bacc7d25e6d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:50.195849 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:50.246513 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75c56ab86ca415dabb82bacc7d25e6d', 'request_id': 'req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:50.255815 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75c56ab86ca415dabb82bacc7d25e6d', 'request_id': 'req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:50.262044 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75c56ab86ca415dabb82bacc7d25e6d', 'request_id': 'req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:50.273292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b admin admin] The token cache is being invalidate because project d75c56ab86ca415dabb82bacc7d25e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:40:50.273575 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:50.274294 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3d374d4e-d2ef-4fbb-a9dc-9c1103ac918b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:40:50.275831 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1381/2768] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:40:50 2022] DELETE /identity/v3/projects/d75c56ab86ca415dabb82bacc7d25e6d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:40:50.281362 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3daa39-4e92-4e40-b1fa-fc149b6e553d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:50.281645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3daa39-4e92-4e40-b1fa-fc149b6e553d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:50.281911 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3daa39-4e92-4e40-b1fa-fc149b6e553d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:50.302455 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5c3daa39-4e92-4e40-b1fa-fc149b6e553d None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:50.303748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5c3daa39-4e92-4e40-b1fa-fc149b6e553d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:50.364363 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1388/2769] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:50.374921 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cb6a4-b02e-4dac-bbfa-92e7a87a873a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:50.375205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cb6a4-b02e-4dac-bbfa-92e7a87a873a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:50.375497 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cb6a4-b02e-4dac-bbfa-92e7a87a873a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:50.400630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-de5cb6a4-b02e-4dac-bbfa-92e7a87a873a None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:50.401888 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-de5cb6a4-b02e-4dac-bbfa-92e7a87a873a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:50.411495 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1382/2770] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:50.415988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3daa39-4e92-4e40-b1fa-fc149b6e553d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:50.428006 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3daa39-4e92-4e40-b1fa-fc149b6e553d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:50.432736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8161e-cb1b-4269-9c74-a55c998bce1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:50.433380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8161e-cb1b-4269-9c74-a55c998bce1e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:50.433654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8161e-cb1b-4269-9c74-a55c998bce1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:50.433924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8161e-cb1b-4269-9c74-a55c998bce1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:50.434572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8161e-cb1b-4269-9c74-a55c998bce1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:40:50.434871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8161e-cb1b-4269-9c74-a55c998bce1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:50.436470 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8161e-cb1b-4269-9c74-a55c998bce1e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:50.448564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1389/2771] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:40:50.453186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cb6a4-b02e-4dac-bbfa-92e7a87a873a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:50.463308 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cb6a4-b02e-4dac-bbfa-92e7a87a873a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:50.468660 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165afa03-5df3-4ea4-a903-986624fb0eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:50.469289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165afa03-5df3-4ea4-a903-986624fb0eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:50.469550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165afa03-5df3-4ea4-a903-986624fb0eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:50.469842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165afa03-5df3-4ea4-a903-986624fb0eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:50.470467 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165afa03-5df3-4ea4-a903-986624fb0eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:50.472343 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165afa03-5df3-4ea4-a903-986624fb0eaf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:50.495349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:50.503381 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1383/2772] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:40:50 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:50.507698 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8161e-cb1b-4269-9c74-a55c998bce1e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:50.518103 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8161e-cb1b-4269-9c74-a55c998bce1e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:50.577099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f614ee5-66dc-4f8b-a96d-b2fa218ad82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:50.577760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f614ee5-66dc-4f8b-a96d-b2fa218ad82b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:50.578013 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f614ee5-66dc-4f8b-a96d-b2fa218ad82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:50.578282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f614ee5-66dc-4f8b-a96d-b2fa218ad82b admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:50.578904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f614ee5-66dc-4f8b-a96d-b2fa218ad82b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:50.580806 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f614ee5-66dc-4f8b-a96d-b2fa218ad82b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:50.605978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1390/2773] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:40:50 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:40:50.610354 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165afa03-5df3-4ea4-a903-986624fb0eaf admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:50.622124 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165afa03-5df3-4ea4-a903-986624fb0eaf admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:50.627134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45c7f2-4796-4004-9d49-bbd48e603282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:50.627914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45c7f2-4796-4004-9d49-bbd48e603282 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:50.628212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45c7f2-4796-4004-9d49-bbd48e603282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:50.628631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45c7f2-4796-4004-9d49-bbd48e603282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:50.629511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45c7f2-4796-4004-9d49-bbd48e603282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:50.631363 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45c7f2-4796-4004-9d49-bbd48e603282 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:50.642324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1384/2774] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:40:50.646587 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f614ee5-66dc-4f8b-a96d-b2fa218ad82b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:50.656950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f614ee5-66dc-4f8b-a96d-b2fa218ad82b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:50.663267 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f722415-9eaa-42dd-a04b-21d64f17578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:50.663991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722415-9eaa-42dd-a04b-21d64f17578b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:50.665115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722415-9eaa-42dd-a04b-21d64f17578b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:50.665115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722415-9eaa-42dd-a04b-21d64f17578b admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:50.665335 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f722415-9eaa-42dd-a04b-21d64f17578b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:50.666810 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f722415-9eaa-42dd-a04b-21d64f17578b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:50.677978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1391/2775] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:40:50.682296 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45c7f2-4796-4004-9d49-bbd48e603282 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:50.692342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45c7f2-4796-4004-9d49-bbd48e603282 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:50.697366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a1011-5948-46e2-a5af-c8748e32b6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:50.698012 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a1011-5948-46e2-a5af-c8748e32b6ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:50.698283 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a1011-5948-46e2-a5af-c8748e32b6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:50.698591 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a1011-5948-46e2-a5af-c8748e32b6ff admin admin] PATH_INFO: `/v3/projects/582739c72b5543b481470876b9d3ffe0/users/0d2660142f0b45339be1882fb015f0b8/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:50.713813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a1011-5948-46e2-a5af-c8748e32b6ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=582739c72b5543b481470876b9d3ffe0, user_id=0d2660142f0b45339be1882fb015f0b8, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:50.716431 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a1011-5948-46e2-a5af-c8748e32b6ff admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:50.729749 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1385/2776] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:40:50 2022] PUT /identity/v3/projects/582739c72b5543b481470876b9d3ffe0/users/0d2660142f0b45339be1882fb015f0b8/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:40:50.741351 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587d0e8-a12b-482f-939c-e41b7d03aca8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:50.741602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587d0e8-a12b-482f-939c-e41b7d03aca8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:50.741872 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587d0e8-a12b-482f-939c-e41b7d03aca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:50.750155 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:50.773177 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9587d0e8-a12b-482f-939c-e41b7d03aca8 None None] MFA Rules not processed for user `0d2660142f0b45339be1882fb015f0b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:40:50.774412 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9587d0e8-a12b-482f-939c-e41b7d03aca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:50.843378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1392/2777] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:40:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:40:50.855228 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a1011-5948-46e2-a5af-c8748e32b6ff admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:40:50.857224 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5b2a1011-5948-46e2-a5af-c8748e32b6ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:40:50.939611 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a1011-5948-46e2-a5af-c8748e32b6ff admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:40:50.944545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1cbe74-b34c-4bf4-80e1-05dd0e4eb93c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf8a0ae03c94aa78cc2ab0e4e1c87b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:40:50.945196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cbe74-b34c-4bf4-80e1-05dd0e4eb93c service glance] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:40:50.945479 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cbe74-b34c-4bf4-80e1-05dd0e4eb93c service glance] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:40:50.945800 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cbe74-b34c-4bf4-80e1-05dd0e4eb93c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:40:50.955125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1cbe74-b34c-4bf4-80e1-05dd0e4eb93c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:40:50.956550 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1cbe74-b34c-4bf4-80e1-05dd0e4eb93c service glance] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:40:50.966509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1386/2778] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:40:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:41:00.840466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=30 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:00.961495 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=51,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:03.902560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587d0e8-a12b-482f-939c-e41b7d03aca8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:41:03.904552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9587d0e8-a12b-482f-939c-e41b7d03aca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:03.967376 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587d0e8-a12b-482f-939c-e41b7d03aca8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:41:03.971376 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841972e6-3c67-41ec-a000-384ffbea8895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:41:03.971967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841972e6-3c67-41ec-a000-384ffbea8895 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:41:03.972237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841972e6-3c67-41ec-a000-384ffbea8895 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:41:03.972499 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841972e6-3c67-41ec-a000-384ffbea8895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:41:03.974458 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-841972e6-3c67-41ec-a000-384ffbea8895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:04.059198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841972e6-3c67-41ec-a000-384ffbea8895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:41:04.060628 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841972e6-3c67-41ec-a000-384ffbea8895 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:41:04.080425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1393/2779] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:41:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:41:14.077122 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=61 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:22.957602 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1cbe74-b34c-4bf4-80e1-05dd0e4eb93c service glance] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:41:22.959082 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0a1cbe74-b34c-4bf4-80e1-05dd0e4eb93c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:23.042616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1cbe74-b34c-4bf4-80e1-05dd0e4eb93c service glance] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:41:23.048464 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fe690-64db-4807-b0f8-da6213b074ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:41:23.049120 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fe690-64db-4807-b0f8-da6213b074ac service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:41:23.049397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fe690-64db-4807-b0f8-da6213b074ac service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:41:23.049672 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fe690-64db-4807-b0f8-da6213b074ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:41:23.051425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-319fe690-64db-4807-b0f8-da6213b074ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:23.064239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fe690-64db-4807-b0f8-da6213b074ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:41:23.065869 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fe690-64db-4807-b0f8-da6213b074ac service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:41:23.076240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1387/2780] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:41:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:41:25.998957 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841972e6-3c67-41ec-a000-384ffbea8895 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:41:26.000494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-841972e6-3c67-41ec-a000-384ffbea8895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:26.011798 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841972e6-3c67-41ec-a000-384ffbea8895 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:41:26.015558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da879b5-30ac-443c-aee7-d14b59dd6263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:41:26.016159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da879b5-30ac-443c-aee7-d14b59dd6263 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:41:26.016405 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da879b5-30ac-443c-aee7-d14b59dd6263 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:41:26.016700 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da879b5-30ac-443c-aee7-d14b59dd6263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:41:26.018496 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0da879b5-30ac-443c-aee7-d14b59dd6263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:26.080510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da879b5-30ac-443c-aee7-d14b59dd6263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:41:26.081959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da879b5-30ac-443c-aee7-d14b59dd6263 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:41:26.092131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1394/2781] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:41:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:41:28.449799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fe690-64db-4807-b0f8-da6213b074ac service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:41:28.460991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fe690-64db-4807-b0f8-da6213b074ac service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:41:28.466717 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:41:28.467947 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:41:28.468230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:41:28.468505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] PATH_INFO: `/v3/users/0d2660142f0b45339be1882fb015f0b8` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:41:28.469805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d2660142f0b45339be1882fb015f0b8)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:41:28.471697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:41:28.523911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d2660142f0b45339be1882fb015f0b8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:28.531739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d2660142f0b45339be1882fb015f0b8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:28.532440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d2660142f0b45339be1882fb015f0b8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:28.539954 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d2660142f0b45339be1882fb015f0b8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:28.552129 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d2660142f0b45339be1882fb015f0b8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:28.558312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1388/2782] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:41:28 2022] DELETE /identity/v3/users/0d2660142f0b45339be1882fb015f0b8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:41:28.563343 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e05509-7f75-4479-9f31-29343747763f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:41:28.563858 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e05509-7f75-4479-9f31-29343747763f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:41:28.563958 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e05509-7f75-4479-9f31-29343747763f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:41:28.583812 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-87e05509-7f75-4479-9f31-29343747763f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:41:28.585071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-87e05509-7f75-4479-9f31-29343747763f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:28.647978 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1395/2783] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:41:28.655942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:41:28.657539 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:28.731619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24772f14-53e4-43a9-a6e0-8e5b6c42c3e1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:41:28.735619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73b1ed-01fc-49d5-858b-00fd0cae2e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:41:28.736314 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73b1ed-01fc-49d5-858b-00fd0cae2e46 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:41:28.736665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73b1ed-01fc-49d5-858b-00fd0cae2e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:41:28.737025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73b1ed-01fc-49d5-858b-00fd0cae2e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:41:28.746733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73b1ed-01fc-49d5-858b-00fd0cae2e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:41:28.748175 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73b1ed-01fc-49d5-858b-00fd0cae2e46 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:41:28.755672 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:28.774964 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1389/2784] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:41:29.255126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e05509-7f75-4479-9f31-29343747763f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:41:29.256727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-87e05509-7f75-4479-9f31-29343747763f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:29.274015 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e05509-7f75-4479-9f31-29343747763f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:41:29.279138 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:41:29.280035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:41:29.280302 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:41:29.280621 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] PATH_INFO: `/v3/projects/582739c72b5543b481470876b9d3ffe0` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:41:29.282244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=582739c72b5543b481470876b9d3ffe0)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:41:29.284195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:41:29.336489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '582739c72b5543b481470876b9d3ffe0', 'request_id': 'req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:29.345671 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '582739c72b5543b481470876b9d3ffe0', 'request_id': 'req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:29.352000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '582739c72b5543b481470876b9d3ffe0', 'request_id': 'req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:29.364212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] The token cache is being invalidate because project 582739c72b5543b481470876b9d3ffe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:41:29.364508 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:29.365398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:29.367152 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1396/2785] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:41:29 2022] DELETE /identity/v3/projects/582739c72b5543b481470876b9d3ffe0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:41:32.356830 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73b1ed-01fc-49d5-858b-00fd0cae2e46 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:41:32.358324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2b73b1ed-01fc-49d5-858b-00fd0cae2e46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:32.411963 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:41:32.413509 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:32.436600 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:32.444213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73b1ed-01fc-49d5-858b-00fd0cae2e46 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:41:32.482702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaba187-6949-48dc-bc3f-ddc12cc29d8f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:41:32.487919 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:41:32.488777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:41:32.489105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:41:32.489403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] PATH_INFO: `/v3/users/ff241cbd3f7c47328ac5ef2ed4c66e18` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:41:32.490871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff241cbd3f7c47328ac5ef2ed4c66e18)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:41:32.493198 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:41:32.507225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4246b-ce29-4e05-8bd4-4c174c06d470 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:41:32.507829 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4246b-ce29-4e05-8bd4-4c174c06d470 service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:41:32.508108 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4246b-ce29-4e05-8bd4-4c174c06d470 service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:41:32.508377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4246b-ce29-4e05-8bd4-4c174c06d470 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:41:32.510204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c7f4246b-ce29-4e05-8bd4-4c174c06d470 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:32.522150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4246b-ce29-4e05-8bd4-4c174c06d470 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:41:32.523754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4246b-ce29-4e05-8bd4-4c174c06d470 service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:41:32.535199 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1390/2786] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:41:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:41:32.542809 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff241cbd3f7c47328ac5ef2ed4c66e18'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:32.551144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff241cbd3f7c47328ac5ef2ed4c66e18'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:32.551756 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff241cbd3f7c47328ac5ef2ed4c66e18'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:32.561171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff241cbd3f7c47328ac5ef2ed4c66e18'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:32.574787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff241cbd3f7c47328ac5ef2ed4c66e18'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:32.580584 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1397/2787] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:41:32 2022] DELETE /identity/v3/users/ff241cbd3f7c47328ac5ef2ed4c66e18 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:41:32.766983 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4246b-ce29-4e05-8bd4-4c174c06d470 service placement] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:41:32.778074 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4246b-ce29-4e05-8bd4-4c174c06d470 service placement] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:41:32.835787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:41:32.836403 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:41:32.836677 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:41:32.836946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] PATH_INFO: `/v3/projects/752e65efe679458fab6b8dbc3a9de82f` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:41:32.838483 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=752e65efe679458fab6b8dbc3a9de82f)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:41:32.840387 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:41:32.878589 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:32.889010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '752e65efe679458fab6b8dbc3a9de82f', 'request_id': 'req-05d8cf1c-06f8-4786-8deb-143a0a8fb129'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:32.897257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '752e65efe679458fab6b8dbc3a9de82f', 'request_id': 'req-05d8cf1c-06f8-4786-8deb-143a0a8fb129'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:32.902940 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '752e65efe679458fab6b8dbc3a9de82f', 'request_id': 'req-05d8cf1c-06f8-4786-8deb-143a0a8fb129'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:32.916978 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] The token cache is being invalidate because project 752e65efe679458fab6b8dbc3a9de82f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:41:32.917306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:32.918061 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:32.920976 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1391/2788] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:41:32 2022] DELETE /identity/v3/projects/752e65efe679458fab6b8dbc3a9de82f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:41:32.926454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:41:32.928130 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:32.943655 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da902a-2c08-47af-9fae-b90d1f7012ba admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:41:33.003582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:41:33.004210 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:41:33.004485 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:41:33.004755 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] PATH_INFO: `/v3/users/b1532b271e4d477eabcfaf602b684bd3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:41:33.017789 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1532b271e4d477eabcfaf602b684bd3)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:41:33.019659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:41:33.048895 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:33.068039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1532b271e4d477eabcfaf602b684bd3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:33.077108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1532b271e4d477eabcfaf602b684bd3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:33.077820 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1532b271e4d477eabcfaf602b684bd3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:33.085001 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1532b271e4d477eabcfaf602b684bd3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:33.097300 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1532b271e4d477eabcfaf602b684bd3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:33.103121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1398/2789] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:41:32 2022] DELETE /identity/v3/users/b1532b271e4d477eabcfaf602b684bd3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:41:33.288486 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:41:33.290082 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:41:33.303880 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8cf1c-06f8-4786-8deb-143a0a8fb129 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:41:33.363196 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:41:33.363857 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:41:33.364102 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:41:33.364377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] PATH_INFO: `/v3/projects/7a5abdc0c6ab45338c7c41910f231a30` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:41:33.375488 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a5abdc0c6ab45338c7c41910f231a30)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:41:33.377505 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:41:33.426554 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a5abdc0c6ab45338c7c41910f231a30', 'request_id': 'req-1c5b268a-629c-4d79-b2c2-295c808352a6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:33.434204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a5abdc0c6ab45338c7c41910f231a30', 'request_id': 'req-1c5b268a-629c-4d79-b2c2-295c808352a6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:33.439768 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a5abdc0c6ab45338c7c41910f231a30', 'request_id': 'req-1c5b268a-629c-4d79-b2c2-295c808352a6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:33.451230 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] The token cache is being invalidate because project 7a5abdc0c6ab45338c7c41910f231a30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:41:33.451511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:33.452248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:41:33.453763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1392/2790] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:41:33 2022] DELETE /identity/v3/projects/7a5abdc0c6ab45338c7c41910f231a30 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:41:43.099910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=19,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:43.448216 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=12,keystone:SELECT=66,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:42:38.626045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:42:38.627692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:42:38.702071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae8b31-23d5-4dda-a1ef-efa1e1bd5ea7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:42:38.764110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ced76d-cd7d-4762-b880-ef558102899f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:42:38.764720 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ced76d-cd7d-4762-b880-ef558102899f service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:42:38.765000 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ced76d-cd7d-4762-b880-ef558102899f service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:42:38.765308 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ced76d-cd7d-4762-b880-ef558102899f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:42:38.767219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c7ced76d-cd7d-4762-b880-ef558102899f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:42:38.778653 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ced76d-cd7d-4762-b880-ef558102899f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:42:38.780248 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ced76d-cd7d-4762-b880-ef558102899f service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:42:38.790132 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1399/2791] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:42:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:42:48.786139 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=58 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:43:22.719584 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:43:22.721278 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:43:22.781520 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b268a-629c-4d79-b2c2-295c808352a6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:43:22.785103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924d541-6e6d-453e-9191-e6815d8888e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:43:22.785721 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924d541-6e6d-453e-9191-e6815d8888e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:43:22.786021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924d541-6e6d-453e-9191-e6815d8888e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:43:22.786288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924d541-6e6d-453e-9191-e6815d8888e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:43:22.788238 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2924d541-6e6d-453e-9191-e6815d8888e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:43:22.862633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924d541-6e6d-453e-9191-e6815d8888e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:43:22.864112 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924d541-6e6d-453e-9191-e6815d8888e6 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:43:22.875284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1393/2792] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:43:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:43:24.073738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ced76d-cd7d-4762-b880-ef558102899f service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:43:24.075881 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c7ced76d-cd7d-4762-b880-ef558102899f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:43:24.090488 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ced76d-cd7d-4762-b880-ef558102899f service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:43:24.095255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdb974-ee07-4efd-8e6c-9b4b640ff03e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:43:24.095841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdb974-ee07-4efd-8e6c-9b4b640ff03e service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:43:24.096113 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdb974-ee07-4efd-8e6c-9b4b640ff03e service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:43:24.096385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdb974-ee07-4efd-8e6c-9b4b640ff03e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:43:24.098433 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9bfdb974-ee07-4efd-8e6c-9b4b640ff03e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:43:24.112527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfdb974-ee07-4efd-8e6c-9b4b640ff03e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:43:24.113991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfdb974-ee07-4efd-8e6c-9b4b640ff03e service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:43:24.125164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1400/2793] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:43:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:43:32.870254 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=54 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:43:34.120361 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:44:36.253761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e353c2b-6f91-48be-b4ce-94117327a7f4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:44:36.254443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e353c2b-6f91-48be-b4ce-94117327a7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:44:36.254443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e353c2b-6f91-48be-b4ce-94117327a7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:44:36.274844 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4e353c2b-6f91-48be-b4ce-94117327a7f4 None None] MFA Rules not processed for user `a486a4bb6b724dacbb78a9e0fe6a789e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:44:36.276109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4e353c2b-6f91-48be-b4ce-94117327a7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:44:36.339041 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1394/2794] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:44:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:44:36.674461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdb974-ee07-4efd-8e6c-9b4b640ff03e service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:44:36.683637 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdb974-ee07-4efd-8e6c-9b4b640ff03e service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:44:36.688275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd778071-3232-42c8-8ea7-16072649385d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:44:36.688887 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd778071-3232-42c8-8ea7-16072649385d service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:44:36.689193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd778071-3232-42c8-8ea7-16072649385d service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:44:36.689466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd778071-3232-42c8-8ea7-16072649385d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:44:36.691593 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fd778071-3232-42c8-8ea7-16072649385d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:44:36.701785 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd778071-3232-42c8-8ea7-16072649385d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:44:36.703193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd778071-3232-42c8-8ea7-16072649385d service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:44:36.711770 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1401/2795] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:44:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:44:39.870492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e353c2b-6f91-48be-b4ce-94117327a7f4 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:44:39.880927 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e353c2b-6f91-48be-b4ce-94117327a7f4 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:44:39.884926 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adba52-4cc0-4cf0-a69d-27840524b945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:44:39.885678 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adba52-4cc0-4cf0-a69d-27840524b945 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:44:39.886025 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adba52-4cc0-4cf0-a69d-27840524b945 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:44:39.886364 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adba52-4cc0-4cf0-a69d-27840524b945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:44:39.888280 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-23adba52-4cc0-4cf0-a69d-27840524b945 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:44:39.898802 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adba52-4cc0-4cf0-a69d-27840524b945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:44:39.900250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adba52-4cc0-4cf0-a69d-27840524b945 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:44:39.909121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1395/2796] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:44:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:44:46.707863 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:44:49.905100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=31 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:45:53.746098 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd778071-3232-42c8-8ea7-16072649385d service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:45:53.748454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fd778071-3232-42c8-8ea7-16072649385d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:45:53.769722 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd778071-3232-42c8-8ea7-16072649385d service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:45:53.776842 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b64a43-c5c2-406f-8e03-f074911d694c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:45:53.777463 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b64a43-c5c2-406f-8e03-f074911d694c service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:45:53.777731 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b64a43-c5c2-406f-8e03-f074911d694c service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:45:53.778034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b64a43-c5c2-406f-8e03-f074911d694c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:45:53.780120 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c0b64a43-c5c2-406f-8e03-f074911d694c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:45:53.791249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b64a43-c5c2-406f-8e03-f074911d694c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:45:53.792828 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b64a43-c5c2-406f-8e03-f074911d694c service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:45:53.802251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1402/2797] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:45:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:03.797906 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:26.172382 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adba52-4cc0-4cf0-a69d-27840524b945 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:26.182386 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adba52-4cc0-4cf0-a69d-27840524b945 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:26.186398 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24c299-e5ba-48b7-9fd1-57a4ab979b57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:26.186990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24c299-e5ba-48b7-9fd1-57a4ab979b57 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:26.187273 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24c299-e5ba-48b7-9fd1-57a4ab979b57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:26.187545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24c299-e5ba-48b7-9fd1-57a4ab979b57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:26.189440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9b24c299-e5ba-48b7-9fd1-57a4ab979b57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:26.265636 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24c299-e5ba-48b7-9fd1-57a4ab979b57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:26.267051 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24c299-e5ba-48b7-9fd1-57a4ab979b57 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:26.277106 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1396/2798] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:29.447891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b64a43-c5c2-406f-8e03-f074911d694c service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:29.449519 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c0b64a43-c5c2-406f-8e03-f074911d694c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:29.466327 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b64a43-c5c2-406f-8e03-f074911d694c service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:29.472140 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35380b31-eb8d-49a9-ba82-f7b87c6bebaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:29.472785 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35380b31-eb8d-49a9-ba82-f7b87c6bebaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:29.473102 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35380b31-eb8d-49a9-ba82-f7b87c6bebaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:29.473403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35380b31-eb8d-49a9-ba82-f7b87c6bebaf admin admin] PATH_INFO: `/v3/users/a486a4bb6b724dacbb78a9e0fe6a789e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:29.474610 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35380b31-eb8d-49a9-ba82-f7b87c6bebaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a486a4bb6b724dacbb78a9e0fe6a789e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:29.476478 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35380b31-eb8d-49a9-ba82-f7b87c6bebaf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:29.524703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-35380b31-eb8d-49a9-ba82-f7b87c6bebaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a486a4bb6b724dacbb78a9e0fe6a789e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:29.534662 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-35380b31-eb8d-49a9-ba82-f7b87c6bebaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a486a4bb6b724dacbb78a9e0fe6a789e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:29.535332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-35380b31-eb8d-49a9-ba82-f7b87c6bebaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a486a4bb6b724dacbb78a9e0fe6a789e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:29.542307 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-35380b31-eb8d-49a9-ba82-f7b87c6bebaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a486a4bb6b724dacbb78a9e0fe6a789e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:29.554429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-35380b31-eb8d-49a9-ba82-f7b87c6bebaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a486a4bb6b724dacbb78a9e0fe6a789e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:29.560275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1403/2799] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:46:29 2022] DELETE /identity/v3/users/a486a4bb6b724dacbb78a9e0fe6a789e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:29.745125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24c299-e5ba-48b7-9fd1-57a4ab979b57 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:29.756257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24c299-e5ba-48b7-9fd1-57a4ab979b57 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:29.815665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:29.816298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:29.816578 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:29.816845 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b admin admin] PATH_INFO: `/v3/projects/e4563767772c486f8fbb9ea131771412` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:29.818073 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4563767772c486f8fbb9ea131771412)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:29.819982 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:29.869744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4563767772c486f8fbb9ea131771412', 'request_id': 'req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:29.877890 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4563767772c486f8fbb9ea131771412', 'request_id': 'req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:29.888939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4563767772c486f8fbb9ea131771412', 'request_id': 'req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:29.907912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b admin admin] The token cache is being invalidate because project e4563767772c486f8fbb9ea131771412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:46:29.908275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:29.909001 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8f3765bc-eba0-4e90-bbf2-98180bf72a8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:29.910661 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1397/2800] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:46:29 2022] DELETE /identity/v3/projects/e4563767772c486f8fbb9ea131771412 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:29.915670 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ec294-6343-4e2c-9128-0eb192fceff0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:29.915920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ec294-6343-4e2c-9128-0eb192fceff0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:29.916868 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ec294-6343-4e2c-9128-0eb192fceff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:29.941552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-224ec294-6343-4e2c-9128-0eb192fceff0 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:29.942820 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-224ec294-6343-4e2c-9128-0eb192fceff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:30.006725 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1404/2801] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:30.014698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e14e6-7827-4e82-a8dc-b774750143a2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.015156 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e14e6-7827-4e82-a8dc-b774750143a2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.015521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e14e6-7827-4e82-a8dc-b774750143a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.037248 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fd2e14e6-7827-4e82-a8dc-b774750143a2 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:30.038787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fd2e14e6-7827-4e82-a8dc-b774750143a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:30.047312 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1398/2802] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:30.051501 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ec294-6343-4e2c-9128-0eb192fceff0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:30.060901 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ec294-6343-4e2c-9128-0eb192fceff0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:30.065514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97d44d-9d19-4323-bc82-81aa65ed7a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:30.066105 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97d44d-9d19-4323-bc82-81aa65ed7a3b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.066374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97d44d-9d19-4323-bc82-81aa65ed7a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.066626 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97d44d-9d19-4323-bc82-81aa65ed7a3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.067340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97d44d-9d19-4323-bc82-81aa65ed7a3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:46:30.067636 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97d44d-9d19-4323-bc82-81aa65ed7a3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:30.069348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97d44d-9d19-4323-bc82-81aa65ed7a3b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:30.080472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1405/2803] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:46:30.085290 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e14e6-7827-4e82-a8dc-b774750143a2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:30.094492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e14e6-7827-4e82-a8dc-b774750143a2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:30.100064 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9981956-d768-4c92-98c1-e852c0e63f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:30.100660 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9981956-d768-4c92-98c1-e852c0e63f8b admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.100936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9981956-d768-4c92-98c1-e852c0e63f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.101250 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9981956-d768-4c92-98c1-e852c0e63f8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.101877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9981956-d768-4c92-98c1-e852c0e63f8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:30.103765 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9981956-d768-4c92-98c1-e852c0e63f8b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:30.120405 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:30.136487 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1399/2804] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:46:30 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:30.140729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97d44d-9d19-4323-bc82-81aa65ed7a3b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:30.150769 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97d44d-9d19-4323-bc82-81aa65ed7a3b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:30.219695 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8aa890-6a39-40a6-b273-95ac15e2a52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:30.220295 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8aa890-6a39-40a6-b273-95ac15e2a52f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.220578 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8aa890-6a39-40a6-b273-95ac15e2a52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.220826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8aa890-6a39-40a6-b273-95ac15e2a52f admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.221527 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8aa890-6a39-40a6-b273-95ac15e2a52f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:30.223439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8aa890-6a39-40a6-b273-95ac15e2a52f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:30.231423 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:30.251249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1406/2805] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:46:30 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:46:30.255619 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9981956-d768-4c92-98c1-e852c0e63f8b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:30.266033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9981956-d768-4c92-98c1-e852c0e63f8b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:30.271137 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca4152-935c-42bb-865b-83780c0125dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:30.271731 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4152-935c-42bb-865b-83780c0125dc admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.272002 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4152-935c-42bb-865b-83780c0125dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.272258 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4152-935c-42bb-865b-83780c0125dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.272889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca4152-935c-42bb-865b-83780c0125dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:30.274523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca4152-935c-42bb-865b-83780c0125dc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:30.284561 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1400/2806] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:30.288628 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8aa890-6a39-40a6-b273-95ac15e2a52f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:30.299168 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8aa890-6a39-40a6-b273-95ac15e2a52f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:30.305019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74949a-32b5-4e92-ada0-40cd7d742c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:30.305649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74949a-32b5-4e92-ada0-40cd7d742c43 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.305928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74949a-32b5-4e92-ada0-40cd7d742c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.306195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74949a-32b5-4e92-ada0-40cd7d742c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.306840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74949a-32b5-4e92-ada0-40cd7d742c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:30.308477 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74949a-32b5-4e92-ada0-40cd7d742c43 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:30.319540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1407/2807] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:30.324212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca4152-935c-42bb-865b-83780c0125dc admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:30.335197 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca4152-935c-42bb-865b-83780c0125dc admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:30.340932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359a1e1-d0a6-41aa-9239-16c9f1371009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:30.341618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359a1e1-d0a6-41aa-9239-16c9f1371009 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.341896 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359a1e1-d0a6-41aa-9239-16c9f1371009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.342167 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359a1e1-d0a6-41aa-9239-16c9f1371009 admin admin] PATH_INFO: `/v3/projects/f2d7e127990645ac8c4b785c252266ae/users/456654160d92475e82947e54a7062b22/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.360941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359a1e1-d0a6-41aa-9239-16c9f1371009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d7e127990645ac8c4b785c252266ae, user_id=456654160d92475e82947e54a7062b22, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:30.364790 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359a1e1-d0a6-41aa-9239-16c9f1371009 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:30.382234 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1401/2808] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:30 2022] PUT /identity/v3/projects/f2d7e127990645ac8c4b785c252266ae/users/456654160d92475e82947e54a7062b22/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:30.392783 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e7442-dd48-4497-a4fd-462ab7acba88 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.393072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e7442-dd48-4497-a4fd-462ab7acba88 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.393374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e7442-dd48-4497-a4fd-462ab7acba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.421175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9d7e7442-dd48-4497-a4fd-462ab7acba88 None None] MFA Rules not processed for user `456654160d92475e82947e54a7062b22`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:30.422415 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9d7e7442-dd48-4497-a4fd-462ab7acba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:30.488779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1408/2809] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:30 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:30.493540 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359a1e1-d0a6-41aa-9239-16c9f1371009 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:30.556482 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359a1e1-d0a6-41aa-9239-16c9f1371009 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:30.562103 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9160410-165a-4801-b811-c86ae9c5bfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:30.562709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9160410-165a-4801-b811-c86ae9c5bfd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.562986 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9160410-165a-4801-b811-c86ae9c5bfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.563253 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9160410-165a-4801-b811-c86ae9c5bfd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.563877 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9160410-165a-4801-b811-c86ae9c5bfd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:30.565999 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9160410-165a-4801-b811-c86ae9c5bfd7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:30.595742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1402/2810] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:46:30 2022] POST /identity/v3/projects => generated 405 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:30.600219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e7442-dd48-4497-a4fd-462ab7acba88 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:30.611353 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e7442-dd48-4497-a4fd-462ab7acba88 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:30.669367 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d99f31-5381-434a-877b-a4ed76e3dfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:30.669993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d99f31-5381-434a-877b-a4ed76e3dfeb admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.670277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d99f31-5381-434a-877b-a4ed76e3dfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.670544 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d99f31-5381-434a-877b-a4ed76e3dfeb admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.671184 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d99f31-5381-434a-877b-a4ed76e3dfeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:30.673219 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d99f31-5381-434a-877b-a4ed76e3dfeb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:30.696592 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1409/2811] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:46:30 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:30.700704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9160410-165a-4801-b811-c86ae9c5bfd7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:30.712010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9160410-165a-4801-b811-c86ae9c5bfd7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:30.717655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5cfe0-5612-4ece-ba41-82a65e593b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:30.718255 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5cfe0-5612-4ece-ba41-82a65e593b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.718524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5cfe0-5612-4ece-ba41-82a65e593b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.718789 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5cfe0-5612-4ece-ba41-82a65e593b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.719433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5cfe0-5612-4ece-ba41-82a65e593b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:30.721026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5cfe0-5612-4ece-ba41-82a65e593b20 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:30.732423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1403/2812] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:30.736676 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d99f31-5381-434a-877b-a4ed76e3dfeb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:30.747280 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d99f31-5381-434a-877b-a4ed76e3dfeb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:30.753309 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9e712-638f-4cd9-b2e5-67a049b4a18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:30.753960 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e712-638f-4cd9-b2e5-67a049b4a18e admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.754246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e712-638f-4cd9-b2e5-67a049b4a18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.754523 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e712-638f-4cd9-b2e5-67a049b4a18e admin admin] PATH_INFO: `/v3/projects/0bdd215baa2345448ecbab00578b1cfd/users/9066aa80d9a34547823cfb610a24208c/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.770074 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9e712-638f-4cd9-b2e5-67a049b4a18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd215baa2345448ecbab00578b1cfd, user_id=9066aa80d9a34547823cfb610a24208c, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:30.772716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9e712-638f-4cd9-b2e5-67a049b4a18e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:30.786566 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1410/2813] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:30 2022] PUT /identity/v3/projects/0bdd215baa2345448ecbab00578b1cfd/users/9066aa80d9a34547823cfb610a24208c/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:30.790671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5cfe0-5612-4ece-ba41-82a65e593b20 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:30.800798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5cfe0-5612-4ece-ba41-82a65e593b20 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:30.862509 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e19b6-cb78-420d-b20a-fb2b1ccc6e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:30.863159 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e19b6-cb78-420d-b20a-fb2b1ccc6e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.863435 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e19b6-cb78-420d-b20a-fb2b1ccc6e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.863701 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e19b6-cb78-420d-b20a-fb2b1ccc6e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.864366 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e19b6-cb78-420d-b20a-fb2b1ccc6e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:30.865994 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e19b6-cb78-420d-b20a-fb2b1ccc6e86 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:30.876570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1404/2814] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:30.882839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9e712-638f-4cd9-b2e5-67a049b4a18e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:30.893213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9e712-638f-4cd9-b2e5-67a049b4a18e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:30.898285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880913a4-4f8a-4b7b-b2d7-56356b28ef61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:30.898928 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880913a4-4f8a-4b7b-b2d7-56356b28ef61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.899193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880913a4-4f8a-4b7b-b2d7-56356b28ef61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.899461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880913a4-4f8a-4b7b-b2d7-56356b28ef61 admin admin] PATH_INFO: `/v3/projects/0bdd215baa2345448ecbab00578b1cfd/users/9066aa80d9a34547823cfb610a24208c/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.901417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880913a4-4f8a-4b7b-b2d7-56356b28ef61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd215baa2345448ecbab00578b1cfd, user_id=9066aa80d9a34547823cfb610a24208c, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:30.904025 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880913a4-4f8a-4b7b-b2d7-56356b28ef61 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:30.914401 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1411/2815] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:30 2022] PUT /identity/v3/projects/0bdd215baa2345448ecbab00578b1cfd/users/9066aa80d9a34547823cfb610a24208c/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:30.925812 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97a2db-6f3d-45f6-a6a5-b41e9752f49d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:30.926097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97a2db-6f3d-45f6-a6a5-b41e9752f49d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:30.926389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97a2db-6f3d-45f6-a6a5-b41e9752f49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:30.959081 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-8b97a2db-6f3d-45f6-a6a5-b41e9752f49d None None] MFA Rules not processed for user `9066aa80d9a34547823cfb610a24208c`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:30.960317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8b97a2db-6f3d-45f6-a6a5-b41e9752f49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:30.978903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:31.034672 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1405/2816] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:30 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:31.046493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880913a4-4f8a-4b7b-b2d7-56356b28ef61 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:31.049658 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-880913a4-4f8a-4b7b-b2d7-56356b28ef61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:31.079522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:31.117130 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880913a4-4f8a-4b7b-b2d7-56356b28ef61 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:31.121033 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb8ea1-2a15-4fa9-850c-5efcacb60f25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:31.121667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb8ea1-2a15-4fa9-850c-5efcacb60f25 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:31.121938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb8ea1-2a15-4fa9-850c-5efcacb60f25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:31.122191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb8ea1-2a15-4fa9-850c-5efcacb60f25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:31.135387 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb8ea1-2a15-4fa9-850c-5efcacb60f25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:31.136809 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb8ea1-2a15-4fa9-850c-5efcacb60f25 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:31.146611 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1412/2817] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:31 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:31.299971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97a2db-6f3d-45f6-a6a5-b41e9752f49d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:31.373306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97a2db-6f3d-45f6-a6a5-b41e9752f49d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:31.378572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:31.379198 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:31.379476 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:31.379747 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] PATH_INFO: `/v3/users/456654160d92475e82947e54a7062b22` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:31.380945 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] RBAC: Authorizing `identity:delete_user(user_id=456654160d92475e82947e54a7062b22)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:31.382892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:31.434372 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '456654160d92475e82947e54a7062b22'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:31.442532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '456654160d92475e82947e54a7062b22'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:31.443144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '456654160d92475e82947e54a7062b22'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:31.450151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '456654160d92475e82947e54a7062b22'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:31.461874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '456654160d92475e82947e54a7062b22'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:31.468446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1406/2818] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:46:31 2022] DELETE /identity/v3/users/456654160d92475e82947e54a7062b22 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:31.473305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135f7af-873f-447b-8a8c-e5685cf7811e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:31.473586 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135f7af-873f-447b-8a8c-e5685cf7811e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:31.473859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135f7af-873f-447b-8a8c-e5685cf7811e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:31.493017 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3135f7af-873f-447b-8a8c-e5685cf7811e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:31.494246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3135f7af-873f-447b-8a8c-e5685cf7811e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:31.553959 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1413/2819] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:31.563874 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:31.565417 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:31.606960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:31.627560 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e27e2-9786-476c-b910-17f96dfacfab admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:31.631551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19645f4-9741-46bf-8d56-30f280ac6422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:31.632153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19645f4-9741-46bf-8d56-30f280ac6422 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:31.632432 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19645f4-9741-46bf-8d56-30f280ac6422 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:31.632698 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19645f4-9741-46bf-8d56-30f280ac6422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:31.642161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19645f4-9741-46bf-8d56-30f280ac6422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:31.643584 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19645f4-9741-46bf-8d56-30f280ac6422 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:31.652947 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1407/2820] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:46:31.936044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135f7af-873f-447b-8a8c-e5685cf7811e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:31.937649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3135f7af-873f-447b-8a8c-e5685cf7811e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:31.950552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135f7af-873f-447b-8a8c-e5685cf7811e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:31.955883 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:31.956623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:31.956795 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:31.957226 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] PATH_INFO: `/v3/projects/f2d7e127990645ac8c4b785c252266ae` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:31.958674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2d7e127990645ac8c4b785c252266ae)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:31.960588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:32.007243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2d7e127990645ac8c4b785c252266ae', 'request_id': 'req-ad879a8a-b9d4-47f9-9917-f2cdf9641505'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.016440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2d7e127990645ac8c4b785c252266ae', 'request_id': 'req-ad879a8a-b9d4-47f9-9917-f2cdf9641505'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.022140 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2d7e127990645ac8c4b785c252266ae', 'request_id': 'req-ad879a8a-b9d4-47f9-9917-f2cdf9641505'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.033128 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] The token cache is being invalidate because project f2d7e127990645ac8c4b785c252266ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:46:32.033449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.034189 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.035782 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1414/2821] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:46:31 2022] DELETE /identity/v3/projects/f2d7e127990645ac8c4b785c252266ae => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:32.040223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19645f4-9741-46bf-8d56-30f280ac6422 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:32.050114 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19645f4-9741-46bf-8d56-30f280ac6422 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:32.115367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f5eb3-57af-48b4-86cf-96b6bc5d97fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:32.115998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f5eb3-57af-48b4-86cf-96b6bc5d97fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:32.116281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f5eb3-57af-48b4-86cf-96b6bc5d97fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:32.116555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f5eb3-57af-48b4-86cf-96b6bc5d97fa admin admin] PATH_INFO: `/v3/users/9066aa80d9a34547823cfb610a24208c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:32.117862 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f5eb3-57af-48b4-86cf-96b6bc5d97fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9066aa80d9a34547823cfb610a24208c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:32.119784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f5eb3-57af-48b4-86cf-96b6bc5d97fa admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:32.168960 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b90f5eb3-57af-48b4-86cf-96b6bc5d97fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9066aa80d9a34547823cfb610a24208c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.178878 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b90f5eb3-57af-48b4-86cf-96b6bc5d97fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9066aa80d9a34547823cfb610a24208c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.179531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b90f5eb3-57af-48b4-86cf-96b6bc5d97fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9066aa80d9a34547823cfb610a24208c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.186930 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b90f5eb3-57af-48b4-86cf-96b6bc5d97fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9066aa80d9a34547823cfb610a24208c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.200416 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b90f5eb3-57af-48b4-86cf-96b6bc5d97fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9066aa80d9a34547823cfb610a24208c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.206991 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1408/2822] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:46:32 2022] DELETE /identity/v3/users/9066aa80d9a34547823cfb610a24208c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:32.484621 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:32.486174 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:32.501320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad879a8a-b9d4-47f9-9917-f2cdf9641505 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:32.549057 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:32.563807 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db55de-064b-48ba-bed5-566ffc692db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:32.564518 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db55de-064b-48ba-bed5-566ffc692db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:32.564799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db55de-064b-48ba-bed5-566ffc692db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:32.565072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db55de-064b-48ba-bed5-566ffc692db9 admin admin] PATH_INFO: `/v3/projects/0bdd215baa2345448ecbab00578b1cfd` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:32.566229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db55de-064b-48ba-bed5-566ffc692db9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bdd215baa2345448ecbab00578b1cfd)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:32.568178 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db55de-064b-48ba-bed5-566ffc692db9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:32.617888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a8db55de-064b-48ba-bed5-566ffc692db9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bdd215baa2345448ecbab00578b1cfd', 'request_id': 'req-a8db55de-064b-48ba-bed5-566ffc692db9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.626365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a8db55de-064b-48ba-bed5-566ffc692db9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bdd215baa2345448ecbab00578b1cfd', 'request_id': 'req-a8db55de-064b-48ba-bed5-566ffc692db9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.632417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a8db55de-064b-48ba-bed5-566ffc692db9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bdd215baa2345448ecbab00578b1cfd', 'request_id': 'req-a8db55de-064b-48ba-bed5-566ffc692db9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.643428 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a8db55de-064b-48ba-bed5-566ffc692db9 admin admin] The token cache is being invalidate because project 0bdd215baa2345448ecbab00578b1cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:46:32.643716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a8db55de-064b-48ba-bed5-566ffc692db9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.644420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a8db55de-064b-48ba-bed5-566ffc692db9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:32.646039 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1415/2823] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:46:32 2022] DELETE /identity/v3/projects/0bdd215baa2345448ecbab00578b1cfd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:32.651304 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1fd4-3593-4f9c-b6a1-b1a84e4a4c86 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:32.651588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1fd4-3593-4f9c-b6a1-b1a84e4a4c86 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:32.651863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1fd4-3593-4f9c-b6a1-b1a84e4a4c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:32.672414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-177b1fd4-3593-4f9c-b6a1-b1a84e4a4c86 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:32.673732 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-177b1fd4-3593-4f9c-b6a1-b1a84e4a4c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:32.725674 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:32.736395 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1409/2824] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:46:32.744512 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c321f-8d44-46f8-be4c-4c04d3a1fe19 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:32.744799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c321f-8d44-46f8-be4c-4c04d3a1fe19 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:32.745071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c321f-8d44-46f8-be4c-4c04d3a1fe19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:32.765936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8c9c321f-8d44-46f8-be4c-4c04d3a1fe19 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:32.767171 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8c9c321f-8d44-46f8-be4c-4c04d3a1fe19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:32.776097 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1416/2825] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:32.780446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b1fd4-3593-4f9c-b6a1-b1a84e4a4c86 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:32.790218 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b1fd4-3593-4f9c-b6a1-b1a84e4a4c86 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:32.795333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c793b-77b8-4f7c-b513-85a2596eec44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:32.795922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c793b-77b8-4f7c-b513-85a2596eec44 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:32.796194 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c793b-77b8-4f7c-b513-85a2596eec44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:32.796459 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c793b-77b8-4f7c-b513-85a2596eec44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:32.797116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c793b-77b8-4f7c-b513-85a2596eec44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:46:32.797415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c793b-77b8-4f7c-b513-85a2596eec44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:32.798989 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c793b-77b8-4f7c-b513-85a2596eec44 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:32.810277 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1410/2826] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:46:32.814738 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c321f-8d44-46f8-be4c-4c04d3a1fe19 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:32.824493 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c321f-8d44-46f8-be4c-4c04d3a1fe19 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:32.830283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd2b0f-4614-47b9-b3a9-26c4a0d31ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:32.830880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd2b0f-4614-47b9-b3a9-26c4a0d31ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:32.831154 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd2b0f-4614-47b9-b3a9-26c4a0d31ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:32.831420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd2b0f-4614-47b9-b3a9-26c4a0d31ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:32.832046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd2b0f-4614-47b9-b3a9-26c4a0d31ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:32.834334 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd2b0f-4614-47b9-b3a9-26c4a0d31ee2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:32.865416 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1417/2827] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:46:32 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:32.869813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c793b-77b8-4f7c-b513-85a2596eec44 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:32.879998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c793b-77b8-4f7c-b513-85a2596eec44 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:32.942675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073c30d-3d06-4b77-a8b7-e9c50660c856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:32.943563 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073c30d-3d06-4b77-a8b7-e9c50660c856 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:32.943892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073c30d-3d06-4b77-a8b7-e9c50660c856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:32.944257 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073c30d-3d06-4b77-a8b7-e9c50660c856 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:32.945126 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073c30d-3d06-4b77-a8b7-e9c50660c856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:32.947731 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073c30d-3d06-4b77-a8b7-e9c50660c856 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:32.971292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1411/2828] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:46:32 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:32.975605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd2b0f-4614-47b9-b3a9-26c4a0d31ee2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:32.986379 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd2b0f-4614-47b9-b3a9-26c4a0d31ee2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:32.991840 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8382a-84bb-4e02-98d4-0d862c774f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:32.992453 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8382a-84bb-4e02-98d4-0d862c774f63 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:32.992720 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8382a-84bb-4e02-98d4-0d862c774f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:32.992980 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8382a-84bb-4e02-98d4-0d862c774f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:32.993659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8382a-84bb-4e02-98d4-0d862c774f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:32.995240 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8382a-84bb-4e02-98d4-0d862c774f63 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:33.005438 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1418/2829] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:33.009414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073c30d-3d06-4b77-a8b7-e9c50660c856 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:33.019528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073c30d-3d06-4b77-a8b7-e9c50660c856 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:33.025038 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8ddae-5576-4a76-b1bb-979b8f53316f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:33.025642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8ddae-5576-4a76-b1bb-979b8f53316f admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:33.025928 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8ddae-5576-4a76-b1bb-979b8f53316f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:33.026195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8ddae-5576-4a76-b1bb-979b8f53316f admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:33.026838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8ddae-5576-4a76-b1bb-979b8f53316f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:33.028461 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8ddae-5576-4a76-b1bb-979b8f53316f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:33.040337 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1412/2830] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:33.046627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8382a-84bb-4e02-98d4-0d862c774f63 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:33.062579 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8382a-84bb-4e02-98d4-0d862c774f63 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:33.071212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04779abe-4fc2-4a90-92a0-d6a73e57876d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:33.072070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04779abe-4fc2-4a90-92a0-d6a73e57876d admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:33.072339 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04779abe-4fc2-4a90-92a0-d6a73e57876d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:33.072605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04779abe-4fc2-4a90-92a0-d6a73e57876d admin admin] PATH_INFO: `/v3/projects/8d7d3518d7bb474cac2c47bb112c4428/users/6b9379dc4adb4ad5a6ae63b31e85e353/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:33.088910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04779abe-4fc2-4a90-92a0-d6a73e57876d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7d3518d7bb474cac2c47bb112c4428, user_id=6b9379dc4adb4ad5a6ae63b31e85e353, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:33.091687 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04779abe-4fc2-4a90-92a0-d6a73e57876d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:33.111068 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1419/2831] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:33 2022] PUT /identity/v3/projects/8d7d3518d7bb474cac2c47bb112c4428/users/6b9379dc4adb4ad5a6ae63b31e85e353/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:33.126371 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3175b-5e34-48e9-a4f5-a45db3afca5f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:33.126653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3175b-5e34-48e9-a4f5-a45db3afca5f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:33.126920 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3175b-5e34-48e9-a4f5-a45db3afca5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:33.155781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a4b3175b-5e34-48e9-a4f5-a45db3afca5f None None] MFA Rules not processed for user `6b9379dc4adb4ad5a6ae63b31e85e353`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:33.156962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a4b3175b-5e34-48e9-a4f5-a45db3afca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:33.232254 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1413/2832] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:33.237014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04779abe-4fc2-4a90-92a0-d6a73e57876d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:33.301828 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04779abe-4fc2-4a90-92a0-d6a73e57876d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:33.307247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710cbdf-7c80-4bb8-b71f-4679e0c12c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:33.307844 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710cbdf-7c80-4bb8-b71f-4679e0c12c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:33.308121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710cbdf-7c80-4bb8-b71f-4679e0c12c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:33.308386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710cbdf-7c80-4bb8-b71f-4679e0c12c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:33.309008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710cbdf-7c80-4bb8-b71f-4679e0c12c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:33.310941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710cbdf-7c80-4bb8-b71f-4679e0c12c98 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:33.333188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:33.341376 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1420/2833] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:46:33 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:33.345455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3175b-5e34-48e9-a4f5-a45db3afca5f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:33.356491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3175b-5e34-48e9-a4f5-a45db3afca5f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:33.404617 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:33.414863 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d689e-78ed-4ba2-93b0-94fa6e668f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:33.415627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d689e-78ed-4ba2-93b0-94fa6e668f44 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:33.415904 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d689e-78ed-4ba2-93b0-94fa6e668f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:33.416171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d689e-78ed-4ba2-93b0-94fa6e668f44 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:33.416792 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d689e-78ed-4ba2-93b0-94fa6e668f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:33.418726 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d689e-78ed-4ba2-93b0-94fa6e668f44 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:33.441346 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1414/2834] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:46:33 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:33.445472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710cbdf-7c80-4bb8-b71f-4679e0c12c98 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:33.455904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710cbdf-7c80-4bb8-b71f-4679e0c12c98 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:33.461837 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8bd5e-0477-43ad-a7a0-ad524e4c0ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:33.462550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8bd5e-0477-43ad-a7a0-ad524e4c0ddc admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:33.462827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8bd5e-0477-43ad-a7a0-ad524e4c0ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:33.463094 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8bd5e-0477-43ad-a7a0-ad524e4c0ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:33.463736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8bd5e-0477-43ad-a7a0-ad524e4c0ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:33.465359 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8bd5e-0477-43ad-a7a0-ad524e4c0ddc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:33.476268 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1421/2835] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:33.480448 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d689e-78ed-4ba2-93b0-94fa6e668f44 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:33.490875 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d689e-78ed-4ba2-93b0-94fa6e668f44 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:33.496484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faf5a2-fea3-4164-9365-bac59b9d12d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:33.497162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faf5a2-fea3-4164-9365-bac59b9d12d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:33.497442 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faf5a2-fea3-4164-9365-bac59b9d12d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:33.497709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faf5a2-fea3-4164-9365-bac59b9d12d3 admin admin] PATH_INFO: `/v3/projects/b88f2a00aa5f46a8b5be0d8198e2c008/users/d52724ff1f774676a8b8b155d006fc6c/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:33.515399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25faf5a2-fea3-4164-9365-bac59b9d12d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b88f2a00aa5f46a8b5be0d8198e2c008, user_id=d52724ff1f774676a8b8b155d006fc6c, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:33.518242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25faf5a2-fea3-4164-9365-bac59b9d12d3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:33.532774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1415/2836] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:33 2022] PUT /identity/v3/projects/b88f2a00aa5f46a8b5be0d8198e2c008/users/d52724ff1f774676a8b8b155d006fc6c/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:33.536679 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8bd5e-0477-43ad-a7a0-ad524e4c0ddc admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:33.546820 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8bd5e-0477-43ad-a7a0-ad524e4c0ddc admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:33.605118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ad1a1-7940-4721-ba50-3cd7f23c86fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:33.605760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ad1a1-7940-4721-ba50-3cd7f23c86fd admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:33.606040 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ad1a1-7940-4721-ba50-3cd7f23c86fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:33.606311 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ad1a1-7940-4721-ba50-3cd7f23c86fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:33.606935 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ad1a1-7940-4721-ba50-3cd7f23c86fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:33.608538 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ad1a1-7940-4721-ba50-3cd7f23c86fd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:33.618497 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1422/2837] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:33.622801 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faf5a2-fea3-4164-9365-bac59b9d12d3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:33.633733 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faf5a2-fea3-4164-9365-bac59b9d12d3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:33.639423 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5ee63-4532-4c94-851f-31e09ed71bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:33.640077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5ee63-4532-4c94-851f-31e09ed71bea admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:33.640376 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5ee63-4532-4c94-851f-31e09ed71bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:33.640627 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5ee63-4532-4c94-851f-31e09ed71bea admin admin] PATH_INFO: `/v3/projects/b88f2a00aa5f46a8b5be0d8198e2c008/users/d52724ff1f774676a8b8b155d006fc6c/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:33.642888 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5ee63-4532-4c94-851f-31e09ed71bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b88f2a00aa5f46a8b5be0d8198e2c008, user_id=d52724ff1f774676a8b8b155d006fc6c, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:33.645555 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5ee63-4532-4c94-851f-31e09ed71bea admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:33.656361 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1416/2838] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:33 2022] PUT /identity/v3/projects/b88f2a00aa5f46a8b5be0d8198e2c008/users/d52724ff1f774676a8b8b155d006fc6c/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:46:33.668061 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6992bb2-b9b5-467d-b0eb-8ac440b2c103 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:33.668425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6992bb2-b9b5-467d-b0eb-8ac440b2c103 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:33.668703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6992bb2-b9b5-467d-b0eb-8ac440b2c103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:33.697972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f6992bb2-b9b5-467d-b0eb-8ac440b2c103 None None] MFA Rules not processed for user `d52724ff1f774676a8b8b155d006fc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:33.699172 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f6992bb2-b9b5-467d-b0eb-8ac440b2c103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:33.767591 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1423/2839] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:33.779355 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5ee63-4532-4c94-851f-31e09ed71bea admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:33.781754 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-46e5ee63-4532-4c94-851f-31e09ed71bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:33.840639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5ee63-4532-4c94-851f-31e09ed71bea admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:33.844933 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6e3ce-57d4-4e40-aaa1-e98b4ae36605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:33.845561 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6e3ce-57d4-4e40-aaa1-e98b4ae36605 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:33.845842 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6e3ce-57d4-4e40-aaa1-e98b4ae36605 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:33.846110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6e3ce-57d4-4e40-aaa1-e98b4ae36605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:33.856217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6e3ce-57d4-4e40-aaa1-e98b4ae36605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:33.857691 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6e3ce-57d4-4e40-aaa1-e98b4ae36605 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:33.867399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1417/2840] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:33.920841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6992bb2-b9b5-467d-b0eb-8ac440b2c103 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:33.922417 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f6992bb2-b9b5-467d-b0eb-8ac440b2c103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:33.935533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6992bb2-b9b5-467d-b0eb-8ac440b2c103 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:33.939831 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dba02-4cad-42f7-8a66-3d6f454a1427 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:33.940457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dba02-4cad-42f7-8a66-3d6f454a1427 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:33.940726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dba02-4cad-42f7-8a66-3d6f454a1427 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:33.940985 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dba02-4cad-42f7-8a66-3d6f454a1427 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:33.997195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dba02-4cad-42f7-8a66-3d6f454a1427 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:33.998799 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dba02-4cad-42f7-8a66-3d6f454a1427 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:34.007502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1424/2841] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:37.383736 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6e3ce-57d4-4e40-aaa1-e98b4ae36605 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:37.458742 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6e3ce-57d4-4e40-aaa1-e98b4ae36605 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:37.464010 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:37.464659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:37.464887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:37.465212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] PATH_INFO: `/v3/users/6b9379dc4adb4ad5a6ae63b31e85e353` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:37.466389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b9379dc4adb4ad5a6ae63b31e85e353)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:37.468332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:37.515554 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b9379dc4adb4ad5a6ae63b31e85e353'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:37.518151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:37.525200 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b9379dc4adb4ad5a6ae63b31e85e353'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:37.526242 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b9379dc4adb4ad5a6ae63b31e85e353'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:37.534067 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b9379dc4adb4ad5a6ae63b31e85e353'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:37.545818 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b9379dc4adb4ad5a6ae63b31e85e353'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:37.551704 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1418/2842] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:46:37 2022] DELETE /identity/v3/users/6b9379dc4adb4ad5a6ae63b31e85e353 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:37.556466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc477a-1722-43be-8a32-e2fccf958142 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:37.556991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc477a-1722-43be-8a32-e2fccf958142 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:37.556991 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc477a-1722-43be-8a32-e2fccf958142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:37.569838 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:37.576641 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-71fc477a-1722-43be-8a32-e2fccf958142 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:37.577899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-71fc477a-1722-43be-8a32-e2fccf958142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:37.638923 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1425/2843] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:37.649918 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:37.652367 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:37.715226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd43b71-8e51-4d8c-ae48-63fd00d02d2c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:37.719388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5debb8-9107-4aff-9048-750640f46e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:37.720004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5debb8-9107-4aff-9048-750640f46e40 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:37.720281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5debb8-9107-4aff-9048-750640f46e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:37.720554 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5debb8-9107-4aff-9048-750640f46e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:37.730574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5debb8-9107-4aff-9048-750640f46e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:37.732004 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5debb8-9107-4aff-9048-750640f46e40 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:37.741633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1419/2844] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:38.028562 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc477a-1722-43be-8a32-e2fccf958142 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:38.030228 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-71fc477a-1722-43be-8a32-e2fccf958142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:38.043229 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc477a-1722-43be-8a32-e2fccf958142 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:38.048772 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:38.049438 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:38.049856 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:38.050131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] PATH_INFO: `/v3/projects/8d7d3518d7bb474cac2c47bb112c4428` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:38.051300 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d7d3518d7bb474cac2c47bb112c4428)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:38.053246 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:38.107162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d7d3518d7bb474cac2c47bb112c4428', 'request_id': 'req-791fe4fd-0a1c-4a29-8972-766a59f4a201'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.116149 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d7d3518d7bb474cac2c47bb112c4428', 'request_id': 'req-791fe4fd-0a1c-4a29-8972-766a59f4a201'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.121743 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d7d3518d7bb474cac2c47bb112c4428', 'request_id': 'req-791fe4fd-0a1c-4a29-8972-766a59f4a201'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.132360 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] The token cache is being invalidate because project 8d7d3518d7bb474cac2c47bb112c4428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:46:38.132640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.133295 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.134773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1426/2845] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:46:38 2022] DELETE /identity/v3/projects/8d7d3518d7bb474cac2c47bb112c4428 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:38.138809 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5debb8-9107-4aff-9048-750640f46e40 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:38.148524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5debb8-9107-4aff-9048-750640f46e40 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:38.205023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470096c-85d2-48f2-87e8-52799ece1ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:38.205668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470096c-85d2-48f2-87e8-52799ece1ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:38.205941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470096c-85d2-48f2-87e8-52799ece1ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:38.206205 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470096c-85d2-48f2-87e8-52799ece1ede admin admin] PATH_INFO: `/v3/users/d52724ff1f774676a8b8b155d006fc6c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:38.207332 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470096c-85d2-48f2-87e8-52799ece1ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52724ff1f774676a8b8b155d006fc6c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:38.209614 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470096c-85d2-48f2-87e8-52799ece1ede admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:38.263307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8470096c-85d2-48f2-87e8-52799ece1ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52724ff1f774676a8b8b155d006fc6c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.272284 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8470096c-85d2-48f2-87e8-52799ece1ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52724ff1f774676a8b8b155d006fc6c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.273021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8470096c-85d2-48f2-87e8-52799ece1ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52724ff1f774676a8b8b155d006fc6c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.280063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8470096c-85d2-48f2-87e8-52799ece1ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52724ff1f774676a8b8b155d006fc6c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.286858 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:38.293000 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8470096c-85d2-48f2-87e8-52799ece1ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52724ff1f774676a8b8b155d006fc6c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.299558 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1420/2846] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:46:38 2022] DELETE /identity/v3/users/d52724ff1f774676a8b8b155d006fc6c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:46:38.610037 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:38.611564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:38.624888 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fe4fd-0a1c-4a29-8972-766a59f4a201 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:38.688826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:38.689503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:38.689777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:38.690045 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae admin admin] PATH_INFO: `/v3/projects/b88f2a00aa5f46a8b5be0d8198e2c008` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:38.691242 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=b88f2a00aa5f46a8b5be0d8198e2c008)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:38.693182 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:38.734440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:38.753596 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b88f2a00aa5f46a8b5be0d8198e2c008', 'request_id': 'req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.766265 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b88f2a00aa5f46a8b5be0d8198e2c008', 'request_id': 'req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.773485 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b88f2a00aa5f46a8b5be0d8198e2c008', 'request_id': 'req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.785739 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae admin admin] The token cache is being invalidate because project b88f2a00aa5f46a8b5be0d8198e2c008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:46:38.786108 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.787011 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-68dacb35-ae4b-4c4d-a8c2-9390e1fe59ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:38.790424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1427/2847] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:46:38 2022] DELETE /identity/v3/projects/b88f2a00aa5f46a8b5be0d8198e2c008 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:38.795738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4abde-aa7a-44a0-a084-250f531789a6 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:38.796219 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4abde-aa7a-44a0-a084-250f531789a6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:38.796358 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4abde-aa7a-44a0-a084-250f531789a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:38.817220 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-06d4abde-aa7a-44a0-a084-250f531789a6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:38.818410 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-06d4abde-aa7a-44a0-a084-250f531789a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:38.879496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1421/2848] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:38.888183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40a0e-09be-4053-8f32-c2fcb6e3aa87 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:38.888466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40a0e-09be-4053-8f32-c2fcb6e3aa87 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:38.888736 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40a0e-09be-4053-8f32-c2fcb6e3aa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:38.909163 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-09d40a0e-09be-4053-8f32-c2fcb6e3aa87 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:38.910411 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-09d40a0e-09be-4053-8f32-c2fcb6e3aa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:38.919176 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1428/2849] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:38.923520 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4abde-aa7a-44a0-a084-250f531789a6 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:38.933655 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4abde-aa7a-44a0-a084-250f531789a6 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:38.939528 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961065b4-1716-473e-86bf-814d7fb0e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:38.940104 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961065b4-1716-473e-86bf-814d7fb0e16c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:38.940398 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961065b4-1716-473e-86bf-814d7fb0e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:38.940631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961065b4-1716-473e-86bf-814d7fb0e16c admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:38.941331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961065b4-1716-473e-86bf-814d7fb0e16c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:46:38.941631 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961065b4-1716-473e-86bf-814d7fb0e16c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:38.943208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961065b4-1716-473e-86bf-814d7fb0e16c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:38.955404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1422/2850] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:46:38.960244 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d40a0e-09be-4053-8f32-c2fcb6e3aa87 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:38.969808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d40a0e-09be-4053-8f32-c2fcb6e3aa87 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:38.975203 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d702d4-a09d-4618-9d5c-b597564893d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:38.975800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d702d4-a09d-4618-9d5c-b597564893d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:38.976070 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d702d4-a09d-4618-9d5c-b597564893d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:38.976337 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d702d4-a09d-4618-9d5c-b597564893d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:38.976962 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d702d4-a09d-4618-9d5c-b597564893d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:38.978999 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d702d4-a09d-4618-9d5c-b597564893d4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:39.007882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1429/2851] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:46:38 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:39.012161 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961065b4-1716-473e-86bf-814d7fb0e16c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:39.021959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961065b4-1716-473e-86bf-814d7fb0e16c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:39.081254 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb977e8a-cf4f-4db3-8c1c-ac9ce103fc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:39.081777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977e8a-cf4f-4db3-8c1c-ac9ce103fc69 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:39.082056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977e8a-cf4f-4db3-8c1c-ac9ce103fc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:39.082321 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977e8a-cf4f-4db3-8c1c-ac9ce103fc69 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:39.083288 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb977e8a-cf4f-4db3-8c1c-ac9ce103fc69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:39.086632 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb977e8a-cf4f-4db3-8c1c-ac9ce103fc69 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:39.110378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1423/2852] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:46:39 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:39.116300 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d702d4-a09d-4618-9d5c-b597564893d4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:39.127432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d702d4-a09d-4618-9d5c-b597564893d4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:39.132709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57220b01-7f97-4aff-b003-31e6d9647571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:39.133558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57220b01-7f97-4aff-b003-31e6d9647571 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:39.133688 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57220b01-7f97-4aff-b003-31e6d9647571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:39.133879 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57220b01-7f97-4aff-b003-31e6d9647571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:39.134672 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57220b01-7f97-4aff-b003-31e6d9647571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:39.136288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57220b01-7f97-4aff-b003-31e6d9647571 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:39.149056 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1430/2853] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:39.153344 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb977e8a-cf4f-4db3-8c1c-ac9ce103fc69 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:39.163936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb977e8a-cf4f-4db3-8c1c-ac9ce103fc69 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:39.169807 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f9ec0-29c9-46b2-9613-7cb1146d0fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:39.170399 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f9ec0-29c9-46b2-9613-7cb1146d0fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:39.170670 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f9ec0-29c9-46b2-9613-7cb1146d0fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:39.170932 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f9ec0-29c9-46b2-9613-7cb1146d0fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:39.171566 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f9ec0-29c9-46b2-9613-7cb1146d0fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:39.173170 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f9ec0-29c9-46b2-9613-7cb1146d0fc7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:39.184273 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1424/2854] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:39.188540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57220b01-7f97-4aff-b003-31e6d9647571 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:39.201740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57220b01-7f97-4aff-b003-31e6d9647571 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:39.208159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5fd58-6682-49f4-ac36-fe38cc4ba54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:39.208807 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5fd58-6682-49f4-ac36-fe38cc4ba54e admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:39.209118 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5fd58-6682-49f4-ac36-fe38cc4ba54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:39.209428 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5fd58-6682-49f4-ac36-fe38cc4ba54e admin admin] PATH_INFO: `/v3/projects/f2c250b0d2d94e848579a567241b7e5c/users/ec5f3c2fcc39428fa094acbd1c4f2789/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:39.224492 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5fd58-6682-49f4-ac36-fe38cc4ba54e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c250b0d2d94e848579a567241b7e5c, user_id=ec5f3c2fcc39428fa094acbd1c4f2789, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:39.227158 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5fd58-6682-49f4-ac36-fe38cc4ba54e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:39.245400 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1431/2855] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:39 2022] PUT /identity/v3/projects/f2c250b0d2d94e848579a567241b7e5c/users/ec5f3c2fcc39428fa094acbd1c4f2789/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:46:39.261121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95241b-4a0f-47c8-8adb-8bf6dc9a1efb None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:39.261425 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95241b-4a0f-47c8-8adb-8bf6dc9a1efb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:39.261715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95241b-4a0f-47c8-8adb-8bf6dc9a1efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:39.290871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ed95241b-4a0f-47c8-8adb-8bf6dc9a1efb None None] MFA Rules not processed for user `ec5f3c2fcc39428fa094acbd1c4f2789`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:39.292080 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ed95241b-4a0f-47c8-8adb-8bf6dc9a1efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:39.338743 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:39.362073 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1425/2856] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:39.368703 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5fd58-6682-49f4-ac36-fe38cc4ba54e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:39.435561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5fd58-6682-49f4-ac36-fe38cc4ba54e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:39.440716 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0a027-5161-4070-a375-87bea0a43cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:39.441365 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0a027-5161-4070-a375-87bea0a43cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:39.441646 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0a027-5161-4070-a375-87bea0a43cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:39.441920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0a027-5161-4070-a375-87bea0a43cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:39.442533 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0a027-5161-4070-a375-87bea0a43cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:39.444439 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0a027-5161-4070-a375-87bea0a43cc5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:39.473990 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1432/2857] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:46:39 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:39.478324 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95241b-4a0f-47c8-8adb-8bf6dc9a1efb None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:39.489443 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95241b-4a0f-47c8-8adb-8bf6dc9a1efb None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:39.549340 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b29f5-3f29-4669-8d59-2a4e4d4e0e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:39.549948 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b29f5-3f29-4669-8d59-2a4e4d4e0e1c admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:39.550223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b29f5-3f29-4669-8d59-2a4e4d4e0e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:39.550492 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b29f5-3f29-4669-8d59-2a4e4d4e0e1c admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:39.551201 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b29f5-3f29-4669-8d59-2a4e4d4e0e1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:39.553073 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b29f5-3f29-4669-8d59-2a4e4d4e0e1c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:39.579349 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1426/2858] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:46:39 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:39.583839 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0a027-5161-4070-a375-87bea0a43cc5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:39.594997 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0a027-5161-4070-a375-87bea0a43cc5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:39.599777 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d138e-32c9-42f4-b4fe-05aeecd6c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:39.600380 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d138e-32c9-42f4-b4fe-05aeecd6c30e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:39.600657 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d138e-32c9-42f4-b4fe-05aeecd6c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:39.600927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d138e-32c9-42f4-b4fe-05aeecd6c30e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:39.601603 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d138e-32c9-42f4-b4fe-05aeecd6c30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:39.603225 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d138e-32c9-42f4-b4fe-05aeecd6c30e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:39.614993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1433/2859] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:39.619038 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b29f5-3f29-4669-8d59-2a4e4d4e0e1c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:39.629206 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b29f5-3f29-4669-8d59-2a4e4d4e0e1c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:39.634397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46841920-1b89-415f-8842-60bc2b797fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:39.634990 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46841920-1b89-415f-8842-60bc2b797fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:39.635268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46841920-1b89-415f-8842-60bc2b797fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:39.635536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46841920-1b89-415f-8842-60bc2b797fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:39.636187 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46841920-1b89-415f-8842-60bc2b797fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:39.637786 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46841920-1b89-415f-8842-60bc2b797fb4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:39.647966 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1427/2860] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:39.652119 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d138e-32c9-42f4-b4fe-05aeecd6c30e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:39.662677 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d138e-32c9-42f4-b4fe-05aeecd6c30e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:39.668227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84360bc8-f4ad-413d-ad1d-4c1e9b93c295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:39.668885 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84360bc8-f4ad-413d-ad1d-4c1e9b93c295 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:39.669277 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84360bc8-f4ad-413d-ad1d-4c1e9b93c295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:39.669918 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84360bc8-f4ad-413d-ad1d-4c1e9b93c295 admin admin] PATH_INFO: `/v3/projects/aca7cf35ec9a4ac390400237329bc6f4/users/677fcbb8fabd42e9afce0e3772d9c218/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:39.685821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84360bc8-f4ad-413d-ad1d-4c1e9b93c295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca7cf35ec9a4ac390400237329bc6f4, user_id=677fcbb8fabd42e9afce0e3772d9c218, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:39.688468 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84360bc8-f4ad-413d-ad1d-4c1e9b93c295 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:39.703461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1434/2861] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:39 2022] PUT /identity/v3/projects/aca7cf35ec9a4ac390400237329bc6f4/users/677fcbb8fabd42e9afce0e3772d9c218/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:46:39.770122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bbabf-f956-4f9f-883e-6047163c2a6e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:39.770411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bbabf-f956-4f9f-883e-6047163c2a6e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:39.770740 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bbabf-f956-4f9f-883e-6047163c2a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:39.804973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d97bbabf-f956-4f9f-883e-6047163c2a6e None None] MFA Rules not processed for user `677fcbb8fabd42e9afce0e3772d9c218`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:39.806330 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d97bbabf-f956-4f9f-883e-6047163c2a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:39.873914 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1428/2862] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:39.878550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84360bc8-f4ad-413d-ad1d-4c1e9b93c295 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:39.888131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:39.943317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84360bc8-f4ad-413d-ad1d-4c1e9b93c295 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:39.949414 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6f585-bb4b-4efd-b2e3-3b203881136f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:39.950035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6f585-bb4b-4efd-b2e3-3b203881136f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:39.950298 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6f585-bb4b-4efd-b2e3-3b203881136f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:39.950567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6f585-bb4b-4efd-b2e3-3b203881136f admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:39.951221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6f585-bb4b-4efd-b2e3-3b203881136f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:39.953130 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6f585-bb4b-4efd-b2e3-3b203881136f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:39.982766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1435/2863] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:46:39 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:39.987388 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bbabf-f956-4f9f-883e-6047163c2a6e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:39.998744 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bbabf-f956-4f9f-883e-6047163c2a6e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:40.047885 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:40.059685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa859e3-7d06-4187-9b67-a8f013573595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:40.060292 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa859e3-7d06-4187-9b67-a8f013573595 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:40.060569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa859e3-7d06-4187-9b67-a8f013573595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:40.060838 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa859e3-7d06-4187-9b67-a8f013573595 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:40.061499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa859e3-7d06-4187-9b67-a8f013573595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:40.063439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa859e3-7d06-4187-9b67-a8f013573595 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:40.089480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1429/2864] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:46:39 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:40.093388 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6f585-bb4b-4efd-b2e3-3b203881136f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:40.103758 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6f585-bb4b-4efd-b2e3-3b203881136f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:40.108720 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae704e2c-bec7-45a4-b06b-1270c966919c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:40.109372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae704e2c-bec7-45a4-b06b-1270c966919c admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:40.109692 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae704e2c-bec7-45a4-b06b-1270c966919c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:40.109964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae704e2c-bec7-45a4-b06b-1270c966919c admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:40.110652 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae704e2c-bec7-45a4-b06b-1270c966919c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:40.112288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae704e2c-bec7-45a4-b06b-1270c966919c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:40.123331 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1436/2865] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:40.127446 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa859e3-7d06-4187-9b67-a8f013573595 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:40.137818 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa859e3-7d06-4187-9b67-a8f013573595 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:40.143570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f54f6-ef0f-4a59-9d06-7bd0f91de948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:40.144240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f54f6-ef0f-4a59-9d06-7bd0f91de948 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:40.144515 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f54f6-ef0f-4a59-9d06-7bd0f91de948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:40.144784 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f54f6-ef0f-4a59-9d06-7bd0f91de948 admin admin] PATH_INFO: `/v3/projects/2464be8034ea45b398f06a60a9855682/users/0002f15f0c454aefa19d8c6afe69779d/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:40.161752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f54f6-ef0f-4a59-9d06-7bd0f91de948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2464be8034ea45b398f06a60a9855682, user_id=0002f15f0c454aefa19d8c6afe69779d, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:40.164389 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f54f6-ef0f-4a59-9d06-7bd0f91de948 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:40.180021 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1430/2866] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:40 2022] PUT /identity/v3/projects/2464be8034ea45b398f06a60a9855682/users/0002f15f0c454aefa19d8c6afe69779d/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:40.184251 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae704e2c-bec7-45a4-b06b-1270c966919c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:40.194808 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae704e2c-bec7-45a4-b06b-1270c966919c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:40.261348 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc396b25-9df3-4343-aebb-5bab007c0a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:40.261953 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc396b25-9df3-4343-aebb-5bab007c0a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:40.262233 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc396b25-9df3-4343-aebb-5bab007c0a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:40.262503 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc396b25-9df3-4343-aebb-5bab007c0a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:40.263140 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc396b25-9df3-4343-aebb-5bab007c0a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:40.264766 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc396b25-9df3-4343-aebb-5bab007c0a52 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:40.274429 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1437/2867] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:40.278846 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f54f6-ef0f-4a59-9d06-7bd0f91de948 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:40.289612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f54f6-ef0f-4a59-9d06-7bd0f91de948 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:40.294942 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e631fc-01e6-44d0-bf05-6160a38613b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:40.295606 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e631fc-01e6-44d0-bf05-6160a38613b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:40.295888 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e631fc-01e6-44d0-bf05-6160a38613b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:40.296153 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e631fc-01e6-44d0-bf05-6160a38613b0 admin admin] PATH_INFO: `/v3/projects/2464be8034ea45b398f06a60a9855682/users/0002f15f0c454aefa19d8c6afe69779d/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:40.298240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e631fc-01e6-44d0-bf05-6160a38613b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2464be8034ea45b398f06a60a9855682, user_id=0002f15f0c454aefa19d8c6afe69779d, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:40.300819 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e631fc-01e6-44d0-bf05-6160a38613b0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:40.311934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1431/2868] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:40 2022] PUT /identity/v3/projects/2464be8034ea45b398f06a60a9855682/users/0002f15f0c454aefa19d8c6afe69779d/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:40.322669 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b5c5c-ee2c-4156-b346-9f929f381b04 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:40.322979 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b5c5c-ee2c-4156-b346-9f929f381b04 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:40.323283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b5c5c-ee2c-4156-b346-9f929f381b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:40.355305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-543b5c5c-ee2c-4156-b346-9f929f381b04 None None] MFA Rules not processed for user `0002f15f0c454aefa19d8c6afe69779d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:40.356583 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-543b5c5c-ee2c-4156-b346-9f929f381b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:40.426014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1438/2869] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:40.437534 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e631fc-01e6-44d0-bf05-6160a38613b0 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:40.439579 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-44e631fc-01e6-44d0-bf05-6160a38613b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:40.499583 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e631fc-01e6-44d0-bf05-6160a38613b0 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:40.504060 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1fcd7-ce8c-455d-8b8b-a3c8b0084028 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:40.504661 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1fcd7-ce8c-455d-8b8b-a3c8b0084028 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:40.504937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1fcd7-ce8c-455d-8b8b-a3c8b0084028 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:40.505235 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1fcd7-ce8c-455d-8b8b-a3c8b0084028 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:40.514508 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1fcd7-ce8c-455d-8b8b-a3c8b0084028 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:40.515959 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1fcd7-ce8c-455d-8b8b-a3c8b0084028 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:40.526135 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1432/2870] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:41.038197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b5c5c-ee2c-4156-b346-9f929f381b04 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:41.040212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-543b5c5c-ee2c-4156-b346-9f929f381b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:41.047489 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:41.053625 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b5c5c-ee2c-4156-b346-9f929f381b04 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:41.058349 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3c9d8-fca1-4517-93e8-f756d37e6edb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:41.058952 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3c9d8-fca1-4517-93e8-f756d37e6edb service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:41.059236 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3c9d8-fca1-4517-93e8-f756d37e6edb service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:41.059511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3c9d8-fca1-4517-93e8-f756d37e6edb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:41.119834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3c9d8-fca1-4517-93e8-f756d37e6edb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:41.121291 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3c9d8-fca1-4517-93e8-f756d37e6edb service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:41.130287 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1439/2871] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:45.217383 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1fcd7-ce8c-455d-8b8b-a3c8b0084028 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:45.226504 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1fcd7-ce8c-455d-8b8b-a3c8b0084028 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:45.230904 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851a35c-a5fa-4742-b11b-f09f7f4c8dde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:45.231500 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851a35c-a5fa-4742-b11b-f09f7f4c8dde service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:45.231774 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851a35c-a5fa-4742-b11b-f09f7f4c8dde service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:45.232046 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851a35c-a5fa-4742-b11b-f09f7f4c8dde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:45.290941 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851a35c-a5fa-4742-b11b-f09f7f4c8dde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:45.292377 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851a35c-a5fa-4742-b11b-f09f7f4c8dde service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:45.301652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1433/2872] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:47.455846 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3c9d8-fca1-4517-93e8-f756d37e6edb service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:47.518005 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3c9d8-fca1-4517-93e8-f756d37e6edb service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:47.524234 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:47.524876 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:47.525187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:47.525494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] PATH_INFO: `/v3/users/ec5f3c2fcc39428fa094acbd1c4f2789` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:47.526762 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec5f3c2fcc39428fa094acbd1c4f2789)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:47.528614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:47.579346 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec5f3c2fcc39428fa094acbd1c4f2789'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:47.588405 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec5f3c2fcc39428fa094acbd1c4f2789'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:47.589350 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec5f3c2fcc39428fa094acbd1c4f2789'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:47.600224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec5f3c2fcc39428fa094acbd1c4f2789'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:47.613510 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec5f3c2fcc39428fa094acbd1c4f2789'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:47.619907 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1440/2873] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:46:47 2022] DELETE /identity/v3/users/ec5f3c2fcc39428fa094acbd1c4f2789 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:47.624650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d609ec-6b79-42fb-a528-c640919e28d0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:47.625221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d609ec-6b79-42fb-a528-c640919e28d0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:47.625221 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d609ec-6b79-42fb-a528-c640919e28d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:47.645544 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-22d609ec-6b79-42fb-a528-c640919e28d0 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:47.646696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-22d609ec-6b79-42fb-a528-c640919e28d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:47.690707 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:47.711282 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1434/2874] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:47.721314 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:47.722931 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:47.746461 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:47.784864 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39276f2-9a56-48f9-8441-4e16f168cc21 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:47.788880 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4f65c-7182-4b46-be6a-22d5917293d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:47.789569 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4f65c-7182-4b46-be6a-22d5917293d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:47.789873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4f65c-7182-4b46-be6a-22d5917293d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:47.790144 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4f65c-7182-4b46-be6a-22d5917293d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:47.799491 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4f65c-7182-4b46-be6a-22d5917293d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:47.800924 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4f65c-7182-4b46-be6a-22d5917293d6 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:47.810514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1441/2875] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:48.095630 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d609ec-6b79-42fb-a528-c640919e28d0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:48.097171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-22d609ec-6b79-42fb-a528-c640919e28d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:48.112034 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d609ec-6b79-42fb-a528-c640919e28d0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:48.118875 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:48.119597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:48.120008 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:48.120397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] PATH_INFO: `/v3/projects/f2c250b0d2d94e848579a567241b7e5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:48.121787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c250b0d2d94e848579a567241b7e5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:48.124011 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:48.174778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c250b0d2d94e848579a567241b7e5c', 'request_id': 'req-1137d80b-20d6-4586-ad91-8f8c11244af8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.183314 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c250b0d2d94e848579a567241b7e5c', 'request_id': 'req-1137d80b-20d6-4586-ad91-8f8c11244af8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.189484 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c250b0d2d94e848579a567241b7e5c', 'request_id': 'req-1137d80b-20d6-4586-ad91-8f8c11244af8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.200355 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] The token cache is being invalidate because project f2c250b0d2d94e848579a567241b7e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:46:48.200639 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.201295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.202903 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1435/2876] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:46:48 2022] DELETE /identity/v3/projects/f2c250b0d2d94e848579a567241b7e5c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:48.206976 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4f65c-7182-4b46-be6a-22d5917293d6 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:48.217217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4f65c-7182-4b46-be6a-22d5917293d6 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:48.277050 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:48.277737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:48.278014 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:48.278290 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] PATH_INFO: `/v3/users/677fcbb8fabd42e9afce0e3772d9c218` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:48.279466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=677fcbb8fabd42e9afce0e3772d9c218)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:48.281623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:48.329044 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677fcbb8fabd42e9afce0e3772d9c218'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.338548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677fcbb8fabd42e9afce0e3772d9c218'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.339322 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677fcbb8fabd42e9afce0e3772d9c218'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.346448 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677fcbb8fabd42e9afce0e3772d9c218'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.359983 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677fcbb8fabd42e9afce0e3772d9c218'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.366151 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1442/2877] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:46:48 2022] DELETE /identity/v3/users/677fcbb8fabd42e9afce0e3772d9c218 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:48.659597 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:48.661211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:48.675980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137d80b-20d6-4586-ad91-8f8c11244af8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:48.736357 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:48.736923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:48.737246 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:48.737523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] PATH_INFO: `/v3/projects/aca7cf35ec9a4ac390400237329bc6f4` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:48.738662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca7cf35ec9a4ac390400237329bc6f4)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:48.740574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:48.788572 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca7cf35ec9a4ac390400237329bc6f4', 'request_id': 'req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.797162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca7cf35ec9a4ac390400237329bc6f4', 'request_id': 'req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.805616 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca7cf35ec9a4ac390400237329bc6f4', 'request_id': 'req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.814652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:48.819048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] The token cache is being invalidate because project aca7cf35ec9a4ac390400237329bc6f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:46:48.819353 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.820529 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.822179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1436/2878] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:46:48 2022] DELETE /identity/v3/projects/aca7cf35ec9a4ac390400237329bc6f4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:48.826955 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:48.828765 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:48.842378 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7e7dd-aa4f-4c1c-9454-632ad56ae990 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:48.884640 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:48.904521 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e60ec-d5b3-4e92-bb4c-ddca29ca68a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:48.905269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e60ec-d5b3-4e92-bb4c-ddca29ca68a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:48.905548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e60ec-d5b3-4e92-bb4c-ddca29ca68a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:48.905816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e60ec-d5b3-4e92-bb4c-ddca29ca68a6 admin admin] PATH_INFO: `/v3/users/0002f15f0c454aefa19d8c6afe69779d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:48.906937 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e60ec-d5b3-4e92-bb4c-ddca29ca68a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0002f15f0c454aefa19d8c6afe69779d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:48.908830 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e60ec-d5b3-4e92-bb4c-ddca29ca68a6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:48.958213 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-440e60ec-d5b3-4e92-bb4c-ddca29ca68a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0002f15f0c454aefa19d8c6afe69779d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.966329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-440e60ec-d5b3-4e92-bb4c-ddca29ca68a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0002f15f0c454aefa19d8c6afe69779d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.966972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-440e60ec-d5b3-4e92-bb4c-ddca29ca68a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0002f15f0c454aefa19d8c6afe69779d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.974421 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-440e60ec-d5b3-4e92-bb4c-ddca29ca68a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0002f15f0c454aefa19d8c6afe69779d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.988032 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-440e60ec-d5b3-4e92-bb4c-ddca29ca68a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0002f15f0c454aefa19d8c6afe69779d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:48.995116 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1443/2879] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:46:48 2022] DELETE /identity/v3/users/0002f15f0c454aefa19d8c6afe69779d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:46:49.285633 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:49.287625 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:49.303653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e6fb2-ac9e-4bf7-b664-45c7d1d4346c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:49.365097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e81f89-a711-4b70-8893-7091cb56314a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:49.365753 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e81f89-a711-4b70-8893-7091cb56314a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:49.366033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e81f89-a711-4b70-8893-7091cb56314a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:49.366303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e81f89-a711-4b70-8893-7091cb56314a admin admin] PATH_INFO: `/v3/projects/2464be8034ea45b398f06a60a9855682` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:49.367473 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e81f89-a711-4b70-8893-7091cb56314a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2464be8034ea45b398f06a60a9855682)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:49.369422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e81f89-a711-4b70-8893-7091cb56314a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:49.418075 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-54e81f89-a711-4b70-8893-7091cb56314a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2464be8034ea45b398f06a60a9855682', 'request_id': 'req-54e81f89-a711-4b70-8893-7091cb56314a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:49.427525 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-54e81f89-a711-4b70-8893-7091cb56314a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2464be8034ea45b398f06a60a9855682', 'request_id': 'req-54e81f89-a711-4b70-8893-7091cb56314a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:49.434222 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-54e81f89-a711-4b70-8893-7091cb56314a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2464be8034ea45b398f06a60a9855682', 'request_id': 'req-54e81f89-a711-4b70-8893-7091cb56314a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:49.445186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-54e81f89-a711-4b70-8893-7091cb56314a admin admin] The token cache is being invalidate because project 2464be8034ea45b398f06a60a9855682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:46:49.445480 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-54e81f89-a711-4b70-8893-7091cb56314a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:49.446108 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-54e81f89-a711-4b70-8893-7091cb56314a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:49.447574 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1437/2880] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:46:49 2022] DELETE /identity/v3/projects/2464be8034ea45b398f06a60a9855682 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:49.453635 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89b5ee-5a27-4a25-9005-572c9f4f68cd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:49.453920 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89b5ee-5a27-4a25-9005-572c9f4f68cd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:49.454193 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89b5ee-5a27-4a25-9005-572c9f4f68cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:49.474255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9d89b5ee-5a27-4a25-9005-572c9f4f68cd None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:49.475441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9d89b5ee-5a27-4a25-9005-572c9f4f68cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:49.542898 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1444/2881] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:49.552718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9611361-e9c5-4b47-a23a-eb61fbf49dc6 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:49.553003 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9611361-e9c5-4b47-a23a-eb61fbf49dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:49.553310 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9611361-e9c5-4b47-a23a-eb61fbf49dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:49.573908 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a9611361-e9c5-4b47-a23a-eb61fbf49dc6 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:49.575107 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a9611361-e9c5-4b47-a23a-eb61fbf49dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:49.584053 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1438/2882] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:49.588336 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89b5ee-5a27-4a25-9005-572c9f4f68cd None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:49.598255 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89b5ee-5a27-4a25-9005-572c9f4f68cd None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:49.603747 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee4709-f095-4cbb-8ca6-187c5d6886eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:49.604345 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee4709-f095-4cbb-8ca6-187c5d6886eb admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:49.604614 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee4709-f095-4cbb-8ca6-187c5d6886eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:49.604882 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee4709-f095-4cbb-8ca6-187c5d6886eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:49.605572 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee4709-f095-4cbb-8ca6-187c5d6886eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:46:49.605902 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee4709-f095-4cbb-8ca6-187c5d6886eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:49.607484 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee4709-f095-4cbb-8ca6-187c5d6886eb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:49.619483 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1445/2883] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:46:49.623999 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9611361-e9c5-4b47-a23a-eb61fbf49dc6 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:49.633179 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9611361-e9c5-4b47-a23a-eb61fbf49dc6 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:49.638210 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d1b78-620b-4434-9d37-9515036f1bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:49.638805 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d1b78-620b-4434-9d37-9515036f1bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:49.639077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d1b78-620b-4434-9d37-9515036f1bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:49.639397 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d1b78-620b-4434-9d37-9515036f1bf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:49.640032 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d1b78-620b-4434-9d37-9515036f1bf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:49.641974 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d1b78-620b-4434-9d37-9515036f1bf2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:49.672736 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1439/2884] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:46:49 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:46:49.677207 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee4709-f095-4cbb-8ca6-187c5d6886eb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:49.687602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee4709-f095-4cbb-8ca6-187c5d6886eb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:49.732586 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:49.747373 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d70412-db60-4638-86dc-e1be2977c8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:49.747977 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d70412-db60-4638-86dc-e1be2977c8a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:49.748247 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d70412-db60-4638-86dc-e1be2977c8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:49.748511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d70412-db60-4638-86dc-e1be2977c8a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:49.749165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d70412-db60-4638-86dc-e1be2977c8a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:49.751074 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d70412-db60-4638-86dc-e1be2977c8a4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:49.773816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1446/2885] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:46:49 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:49.779339 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d1b78-620b-4434-9d37-9515036f1bf2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:49.789985 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d1b78-620b-4434-9d37-9515036f1bf2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:49.795873 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7401987-e659-4b74-83c7-9cd338598a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:49.796439 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7401987-e659-4b74-83c7-9cd338598a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:49.796724 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7401987-e659-4b74-83c7-9cd338598a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:49.796996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7401987-e659-4b74-83c7-9cd338598a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:49.797707 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7401987-e659-4b74-83c7-9cd338598a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:49.799306 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7401987-e659-4b74-83c7-9cd338598a55 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:49.809648 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1440/2886] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:49.813581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d70412-db60-4638-86dc-e1be2977c8a4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:49.823454 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d70412-db60-4638-86dc-e1be2977c8a4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:49.828187 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5156196-1683-4552-b392-ea63671b25ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:49.828775 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5156196-1683-4552-b392-ea63671b25ff admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:49.829030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5156196-1683-4552-b392-ea63671b25ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:49.829321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5156196-1683-4552-b392-ea63671b25ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:49.829961 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5156196-1683-4552-b392-ea63671b25ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:49.831534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5156196-1683-4552-b392-ea63671b25ff admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:49.842514 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1447/2887] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:49.846708 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7401987-e659-4b74-83c7-9cd338598a55 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:49.856979 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7401987-e659-4b74-83c7-9cd338598a55 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:49.863013 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93752f32-ee69-4dd6-a4c2-71b55e7dfba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:49.863667 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93752f32-ee69-4dd6-a4c2-71b55e7dfba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:49.863943 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93752f32-ee69-4dd6-a4c2-71b55e7dfba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:49.864213 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93752f32-ee69-4dd6-a4c2-71b55e7dfba9 admin admin] PATH_INFO: `/v3/projects/5a1903cd863f41a3b960f8e6cf827883/users/7b757d5408084e9ba423d443bec6de9f/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:49.879194 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93752f32-ee69-4dd6-a4c2-71b55e7dfba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1903cd863f41a3b960f8e6cf827883, user_id=7b757d5408084e9ba423d443bec6de9f, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:49.881882 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93752f32-ee69-4dd6-a4c2-71b55e7dfba9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:49.895882 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1441/2888] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:49 2022] PUT /identity/v3/projects/5a1903cd863f41a3b960f8e6cf827883/users/7b757d5408084e9ba423d443bec6de9f/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:49.908208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f375a-b037-46e6-9dab-a060c4732fa1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:49.908585 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f375a-b037-46e6-9dab-a060c4732fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:49.908926 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f375a-b037-46e6-9dab-a060c4732fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:49.940780 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-696f375a-b037-46e6-9dab-a060c4732fa1 None None] MFA Rules not processed for user `7b757d5408084e9ba423d443bec6de9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:49.942030 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-696f375a-b037-46e6-9dab-a060c4732fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:50.011263 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1448/2889] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:46:50.017272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93752f32-ee69-4dd6-a4c2-71b55e7dfba9 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:50.048659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:50.083656 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93752f32-ee69-4dd6-a4c2-71b55e7dfba9 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:50.088871 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3478c54-2639-4d65-aa4d-90c8784ab64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:50.089498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478c54-2639-4d65-aa4d-90c8784ab64b admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:50.089772 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478c54-2639-4d65-aa4d-90c8784ab64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:50.090048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478c54-2639-4d65-aa4d-90c8784ab64b admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:50.090657 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3478c54-2639-4d65-aa4d-90c8784ab64b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:50.092533 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3478c54-2639-4d65-aa4d-90c8784ab64b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:50.125162 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1442/2890] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:46:50 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:50.129398 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f375a-b037-46e6-9dab-a060c4732fa1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:50.140384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f375a-b037-46e6-9dab-a060c4732fa1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:50.200321 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb48db-b4aa-42c2-8c39-ea1901b871fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:50.201217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb48db-b4aa-42c2-8c39-ea1901b871fa admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:50.201575 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb48db-b4aa-42c2-8c39-ea1901b871fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:50.201945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb48db-b4aa-42c2-8c39-ea1901b871fa admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:50.202909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb48db-b4aa-42c2-8c39-ea1901b871fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:50.205841 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb48db-b4aa-42c2-8c39-ea1901b871fa admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:50.233784 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1449/2891] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:46:50 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:50.238091 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3478c54-2639-4d65-aa4d-90c8784ab64b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:50.248872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3478c54-2639-4d65-aa4d-90c8784ab64b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:50.254273 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671f154-e200-453a-84e0-abd1856e268c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:50.254881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671f154-e200-453a-84e0-abd1856e268c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:50.255158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671f154-e200-453a-84e0-abd1856e268c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:50.255433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671f154-e200-453a-84e0-abd1856e268c admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:50.256082 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4671f154-e200-453a-84e0-abd1856e268c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:50.257713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4671f154-e200-453a-84e0-abd1856e268c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:50.268854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1443/2892] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:50.273410 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb48db-b4aa-42c2-8c39-ea1901b871fa admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:50.283909 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb48db-b4aa-42c2-8c39-ea1901b871fa admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:50.289396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f8a2a-e6da-441e-93e1-2c4cf67cf389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:50.290055 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f8a2a-e6da-441e-93e1-2c4cf67cf389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:50.290324 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f8a2a-e6da-441e-93e1-2c4cf67cf389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:50.290601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f8a2a-e6da-441e-93e1-2c4cf67cf389 admin admin] PATH_INFO: `/v3/projects/d3cafacb72d74a0ea370293554fd0785/users/ad1582abb19446db94865282338b31ac/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:50.305973 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f8a2a-e6da-441e-93e1-2c4cf67cf389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cafacb72d74a0ea370293554fd0785, user_id=ad1582abb19446db94865282338b31ac, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:50.308715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f8a2a-e6da-441e-93e1-2c4cf67cf389 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:50.324046 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1450/2893] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:50 2022] PUT /identity/v3/projects/d3cafacb72d74a0ea370293554fd0785/users/ad1582abb19446db94865282338b31ac/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:50.328017 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671f154-e200-453a-84e0-abd1856e268c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:50.338889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671f154-e200-453a-84e0-abd1856e268c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:50.395753 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7895c0-c411-44cd-9d1c-ca77c98d02f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:50.396364 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7895c0-c411-44cd-9d1c-ca77c98d02f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:50.396645 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7895c0-c411-44cd-9d1c-ca77c98d02f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:50.396912 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7895c0-c411-44cd-9d1c-ca77c98d02f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:50.397584 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7895c0-c411-44cd-9d1c-ca77c98d02f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:50.399177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7895c0-c411-44cd-9d1c-ca77c98d02f6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:50.408738 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1444/2894] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:50.413320 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f8a2a-e6da-441e-93e1-2c4cf67cf389 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:50.424074 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f8a2a-e6da-441e-93e1-2c4cf67cf389 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:50.429505 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a5bcd-9816-411a-955d-e6984ea9bcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:50.430153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a5bcd-9816-411a-955d-e6984ea9bcd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:50.430434 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a5bcd-9816-411a-955d-e6984ea9bcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:50.430699 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a5bcd-9816-411a-955d-e6984ea9bcd3 admin admin] PATH_INFO: `/v3/projects/d3cafacb72d74a0ea370293554fd0785/users/ad1582abb19446db94865282338b31ac/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:50.432797 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a5bcd-9816-411a-955d-e6984ea9bcd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cafacb72d74a0ea370293554fd0785, user_id=ad1582abb19446db94865282338b31ac, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:50.435441 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a5bcd-9816-411a-955d-e6984ea9bcd3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:50.446581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1451/2895] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:50 2022] PUT /identity/v3/projects/d3cafacb72d74a0ea370293554fd0785/users/ad1582abb19446db94865282338b31ac/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:50.457128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2aa3cd-e1d3-46eb-8ac7-cd9252a97877 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:50.457777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2aa3cd-e1d3-46eb-8ac7-cd9252a97877 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:50.458048 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2aa3cd-e1d3-46eb-8ac7-cd9252a97877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:50.487275 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0a2aa3cd-e1d3-46eb-8ac7-cd9252a97877 None None] MFA Rules not processed for user `ad1582abb19446db94865282338b31ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:50.488527 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0a2aa3cd-e1d3-46eb-8ac7-cd9252a97877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:50.563978 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1445/2896] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:50.579147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a5bcd-9816-411a-955d-e6984ea9bcd3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:50.582644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9d1a5bcd-9816-411a-955d-e6984ea9bcd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:50.601440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:50.658212 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a5bcd-9816-411a-955d-e6984ea9bcd3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:50.662003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e869e23-79cc-47d3-930b-5a10c8fc6978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:50.662621 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e869e23-79cc-47d3-930b-5a10c8fc6978 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:50.662904 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e869e23-79cc-47d3-930b-5a10c8fc6978 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:50.663191 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e869e23-79cc-47d3-930b-5a10c8fc6978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:50.672649 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e869e23-79cc-47d3-930b-5a10c8fc6978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:50.674280 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e869e23-79cc-47d3-930b-5a10c8fc6978 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:50.683654 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1452/2897] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:50.795662 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2aa3cd-e1d3-46eb-8ac7-cd9252a97877 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:50.806415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2aa3cd-e1d3-46eb-8ac7-cd9252a97877 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:50.811057 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63475d-63e1-4899-b958-fe36ecbba95b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adf758d5fb9430189ee9057b4b7ea97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:50.811663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63475d-63e1-4899-b958-fe36ecbba95b service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:50.811920 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63475d-63e1-4899-b958-fe36ecbba95b service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:50.812208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63475d-63e1-4899-b958-fe36ecbba95b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:50.833226 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:50.873777 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63475d-63e1-4899-b958-fe36ecbba95b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:50.875279 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63475d-63e1-4899-b958-fe36ecbba95b service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:50.884739 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1446/2898] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:53.538238 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e869e23-79cc-47d3-930b-5a10c8fc6978 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:53.605340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e869e23-79cc-47d3-930b-5a10c8fc6978 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:53.610288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:53.610927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:53.611208 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:53.611480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] PATH_INFO: `/v3/users/7b757d5408084e9ba423d443bec6de9f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:53.612714 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b757d5408084e9ba423d443bec6de9f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:53.614730 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:53.675253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b757d5408084e9ba423d443bec6de9f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:53.686279 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b757d5408084e9ba423d443bec6de9f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:53.686943 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b757d5408084e9ba423d443bec6de9f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:53.694436 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b757d5408084e9ba423d443bec6de9f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:53.706304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b757d5408084e9ba423d443bec6de9f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:53.712463 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1453/2899] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:46:53 2022] DELETE /identity/v3/users/7b757d5408084e9ba423d443bec6de9f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:53.717148 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271f889-cc59-487c-ac36-81885b863f27 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:53.717673 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271f889-cc59-487c-ac36-81885b863f27 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:53.717781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271f889-cc59-487c-ac36-81885b863f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:53.736915 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7271f889-cc59-487c-ac36-81885b863f27 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:53.738169 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7271f889-cc59-487c-ac36-81885b863f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:53.798511 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1447/2900] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:53.808892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:53.810561 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:53.846165 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:53.871897 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6a594-5838-4867-ba1e-7d80ba5f1676 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:53.875558 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e19ec1-6f8e-41fc-ba7e-ea596c7a64cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:53.876164 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e19ec1-6f8e-41fc-ba7e-ea596c7a64cd service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:53.876435 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e19ec1-6f8e-41fc-ba7e-ea596c7a64cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:53.876701 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e19ec1-6f8e-41fc-ba7e-ea596c7a64cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:53.886153 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e19ec1-6f8e-41fc-ba7e-ea596c7a64cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:53.887563 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e19ec1-6f8e-41fc-ba7e-ea596c7a64cd service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:53.897220 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1454/2901] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:46:54.180333 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271f889-cc59-487c-ac36-81885b863f27 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:54.181860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7271f889-cc59-487c-ac36-81885b863f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:54.194523 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271f889-cc59-487c-ac36-81885b863f27 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:54.200225 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:54.200872 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:54.201173 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:54.201561 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] PATH_INFO: `/v3/projects/5a1903cd863f41a3b960f8e6cf827883` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:54.202799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1903cd863f41a3b960f8e6cf827883)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:54.204710 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:54.254762 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1903cd863f41a3b960f8e6cf827883', 'request_id': 'req-3bad6b1d-751f-407c-a719-17665731c82b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.262454 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1903cd863f41a3b960f8e6cf827883', 'request_id': 'req-3bad6b1d-751f-407c-a719-17665731c82b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.268498 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1903cd863f41a3b960f8e6cf827883', 'request_id': 'req-3bad6b1d-751f-407c-a719-17665731c82b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.279378 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] The token cache is being invalidate because project 5a1903cd863f41a3b960f8e6cf827883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:46:54.279663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.280303 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.281926 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1448/2902] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:46:54 2022] DELETE /identity/v3/projects/5a1903cd863f41a3b960f8e6cf827883 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:54.286069 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e19ec1-6f8e-41fc-ba7e-ea596c7a64cd service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:54.295974 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e19ec1-6f8e-41fc-ba7e-ea596c7a64cd service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:54.356912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7448e-5499-469b-8d12-6eb2e969d527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:54.357540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7448e-5499-469b-8d12-6eb2e969d527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:54.357816 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7448e-5499-469b-8d12-6eb2e969d527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:54.358084 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7448e-5499-469b-8d12-6eb2e969d527 admin admin] PATH_INFO: `/v3/users/ad1582abb19446db94865282338b31ac` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:54.359254 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7448e-5499-469b-8d12-6eb2e969d527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad1582abb19446db94865282338b31ac)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:54.361249 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7448e-5499-469b-8d12-6eb2e969d527 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:54.416836 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4c7448e-5499-469b-8d12-6eb2e969d527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad1582abb19446db94865282338b31ac'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.427539 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4c7448e-5499-469b-8d12-6eb2e969d527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad1582abb19446db94865282338b31ac'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.428542 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4c7448e-5499-469b-8d12-6eb2e969d527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad1582abb19446db94865282338b31ac'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.439605 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4c7448e-5499-469b-8d12-6eb2e969d527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad1582abb19446db94865282338b31ac'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.452801 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4c7448e-5499-469b-8d12-6eb2e969d527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad1582abb19446db94865282338b31ac'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.459667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1455/2903] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:46:54 2022] DELETE /identity/v3/users/ad1582abb19446db94865282338b31ac => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:54.753177 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:54.754821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:54.767937 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad6b1d-751f-407c-a719-17665731c82b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:54.811821 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:54.828887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:54.829556 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:54.829836 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:54.830109 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff admin admin] PATH_INFO: `/v3/projects/d3cafacb72d74a0ea370293554fd0785` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:54.831274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3cafacb72d74a0ea370293554fd0785)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:54.833651 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:54.884232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3cafacb72d74a0ea370293554fd0785', 'request_id': 'req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.892139 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3cafacb72d74a0ea370293554fd0785', 'request_id': 'req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.898041 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3cafacb72d74a0ea370293554fd0785', 'request_id': 'req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.908946 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff admin admin] The token cache is being invalidate because project d3cafacb72d74a0ea370293554fd0785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:46:54.909264 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.909980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-06e1ac6b-9d27-49c6-97a8-2fc9e797d2ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:46:54.911624 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1449/2904] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:46:54 2022] DELETE /identity/v3/projects/d3cafacb72d74a0ea370293554fd0785 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:54.916821 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4aa02-840f-4018-889c-6bb976df8e6f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:54.917385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4aa02-840f-4018-889c-6bb976df8e6f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:54.917385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4aa02-840f-4018-889c-6bb976df8e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:54.938690 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c6a4aa02-840f-4018-889c-6bb976df8e6f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:54.939892 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c6a4aa02-840f-4018-889c-6bb976df8e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:54.990057 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:55.005790 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1456/2905] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:46:55.013681 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4fd7e-8c79-4dbd-a702-f62fa0328b57 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:55.013992 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4fd7e-8c79-4dbd-a702-f62fa0328b57 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:55.014263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4fd7e-8c79-4dbd-a702-f62fa0328b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:55.035413 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-18c4fd7e-8c79-4dbd-a702-f62fa0328b57 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:55.036668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-18c4fd7e-8c79-4dbd-a702-f62fa0328b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:55.045211 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1450/2906] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:55.050388 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4aa02-840f-4018-889c-6bb976df8e6f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:55.060938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4aa02-840f-4018-889c-6bb976df8e6f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:55.065607 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19222dac-2e79-42bd-ab76-824698c58ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:55.066317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19222dac-2e79-42bd-ab76-824698c58ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:55.066595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19222dac-2e79-42bd-ab76-824698c58ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:55.066866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19222dac-2e79-42bd-ab76-824698c58ce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:55.067511 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19222dac-2e79-42bd-ab76-824698c58ce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:46:55.067814 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19222dac-2e79-42bd-ab76-824698c58ce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:55.069455 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19222dac-2e79-42bd-ab76-824698c58ce3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:55.080938 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1457/2907] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:46:55.085569 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4fd7e-8c79-4dbd-a702-f62fa0328b57 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:55.094798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4fd7e-8c79-4dbd-a702-f62fa0328b57 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:55.099763 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50726c4-a4ca-4ea8-aa81-6a2f1a15c6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:55.100319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50726c4-a4ca-4ea8-aa81-6a2f1a15c6a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:55.100671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50726c4-a4ca-4ea8-aa81-6a2f1a15c6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:55.100939 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50726c4-a4ca-4ea8-aa81-6a2f1a15c6a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:55.101596 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50726c4-a4ca-4ea8-aa81-6a2f1a15c6a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:55.103461 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50726c4-a4ca-4ea8-aa81-6a2f1a15c6a7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:55.131713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1451/2908] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:46:55 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:46:55.136053 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19222dac-2e79-42bd-ab76-824698c58ce3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:55.146221 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19222dac-2e79-42bd-ab76-824698c58ce3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:55.206778 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6be38e-a145-49fc-b531-4f91e33eb08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:55.207755 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6be38e-a145-49fc-b531-4f91e33eb08c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:55.208067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6be38e-a145-49fc-b531-4f91e33eb08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:55.208341 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6be38e-a145-49fc-b531-4f91e33eb08c admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:55.208964 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6be38e-a145-49fc-b531-4f91e33eb08c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:55.210988 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6be38e-a145-49fc-b531-4f91e33eb08c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:55.232480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1458/2909] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:46:55 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:46:55.236668 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50726c4-a4ca-4ea8-aa81-6a2f1a15c6a7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:55.246907 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50726c4-a4ca-4ea8-aa81-6a2f1a15c6a7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:55.252055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1423f-d73c-4a86-a7c1-1bda7663d332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:55.252650 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1423f-d73c-4a86-a7c1-1bda7663d332 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:55.252924 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1423f-d73c-4a86-a7c1-1bda7663d332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:55.253223 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1423f-d73c-4a86-a7c1-1bda7663d332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:55.253860 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1423f-d73c-4a86-a7c1-1bda7663d332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:55.255468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1423f-d73c-4a86-a7c1-1bda7663d332 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:55.265713 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1452/2910] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:55.269715 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6be38e-a145-49fc-b531-4f91e33eb08c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:55.279805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6be38e-a145-49fc-b531-4f91e33eb08c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:55.285048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6624c8d-f088-456e-8b76-0917defa2a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:55.285705 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6624c8d-f088-456e-8b76-0917defa2a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:55.285983 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6624c8d-f088-456e-8b76-0917defa2a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:55.286253 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6624c8d-f088-456e-8b76-0917defa2a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:55.286891 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6624c8d-f088-456e-8b76-0917defa2a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:55.288534 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6624c8d-f088-456e-8b76-0917defa2a70 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:55.299020 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1459/2911] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:46:55.303254 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1423f-d73c-4a86-a7c1-1bda7663d332 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:55.313186 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1423f-d73c-4a86-a7c1-1bda7663d332 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:55.317889 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5591f0-3087-44e3-ae7c-028208793179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:55.318524 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5591f0-3087-44e3-ae7c-028208793179 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:55.320263 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5591f0-3087-44e3-ae7c-028208793179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:55.320531 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5591f0-3087-44e3-ae7c-028208793179 admin admin] PATH_INFO: `/v3/projects/4bcaf3ee57fe4f2091b17cc66d2c1aa0/users/7e2be3bf2b2f47b4833986ab66b5dcf9/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:55.335315 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5591f0-3087-44e3-ae7c-028208793179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bcaf3ee57fe4f2091b17cc66d2c1aa0, user_id=7e2be3bf2b2f47b4833986ab66b5dcf9, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:55.337962 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5591f0-3087-44e3-ae7c-028208793179 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:55.352158 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1453/2912] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:46:55 2022] PUT /identity/v3/projects/4bcaf3ee57fe4f2091b17cc66d2c1aa0/users/7e2be3bf2b2f47b4833986ab66b5dcf9/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:46:55.362729 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e0962-6011-4d75-843a-92f02cf0a40e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:55.363008 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e0962-6011-4d75-843a-92f02cf0a40e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:55.363275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e0962-6011-4d75-843a-92f02cf0a40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:55.391313 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b44e0962-6011-4d75-843a-92f02cf0a40e None None] MFA Rules not processed for user `7e2be3bf2b2f47b4833986ab66b5dcf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:46:55.392450 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b44e0962-6011-4d75-843a-92f02cf0a40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:55.456749 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1460/2913] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:46:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:46:55.468841 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5591f0-3087-44e3-ae7c-028208793179 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:46:55.470963 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-af5591f0-3087-44e3-ae7c-028208793179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:46:55.530239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5591f0-3087-44e3-ae7c-028208793179 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:46:55.534036 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53f08a-aab1-4716-8eab-5de4cab1c82c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acf8a0ae03c94aa78cc2ab0e4e1c87b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:46:55.534643 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53f08a-aab1-4716-8eab-5de4cab1c82c service glance] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:46:55.534919 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53f08a-aab1-4716-8eab-5de4cab1c82c service glance] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:46:55.535193 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53f08a-aab1-4716-8eab-5de4cab1c82c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:46:55.543727 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53f08a-aab1-4716-8eab-5de4cab1c82c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:46:55.545178 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53f08a-aab1-4716-8eab-5de4cab1c82c service glance] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:46:55.554718 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1454/2914] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:46:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:47:05.453689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:05.550474 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:17.610601 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e0962-6011-4d75-843a-92f02cf0a40e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:17.728848 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e0962-6011-4d75-843a-92f02cf0a40e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:17.734374 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:17.735003 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:17.735305 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:17.735570 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] PATH_INFO: `/v3/users/7e2be3bf2b2f47b4833986ab66b5dcf9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:17.736825 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e2be3bf2b2f47b4833986ab66b5dcf9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:17.738773 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:17.792540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e2be3bf2b2f47b4833986ab66b5dcf9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:47:17.803334 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e2be3bf2b2f47b4833986ab66b5dcf9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:47:17.804034 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e2be3bf2b2f47b4833986ab66b5dcf9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:47:17.815143 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e2be3bf2b2f47b4833986ab66b5dcf9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:47:17.827711 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e2be3bf2b2f47b4833986ab66b5dcf9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:47:17.834310 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1461/2915] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:47:17 2022] DELETE /identity/v3/users/7e2be3bf2b2f47b4833986ab66b5dcf9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:47:17.838936 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d115b3-1f85-4384-a565-e743236a0f1e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:17.839501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d115b3-1f85-4384-a565-e743236a0f1e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:17.839501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d115b3-1f85-4384-a565-e743236a0f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:17.889240 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-02d115b3-1f85-4384-a565-e743236a0f1e None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:47:17.890455 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-02d115b3-1f85-4384-a565-e743236a0f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:17.952313 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1455/2916] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:47:17.962567 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:17.964099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:18.008256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:18.063899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea3336-d316-4cab-bba9-6dd23b2b13ca admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:18.070873 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1148968-9af0-4745-b99a-82a9595fa4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:18.071481 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1148968-9af0-4745-b99a-82a9595fa4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:18.071748 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1148968-9af0-4745-b99a-82a9595fa4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:18.072067 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1148968-9af0-4745-b99a-82a9595fa4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:18.082203 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1148968-9af0-4745-b99a-82a9595fa4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:18.083633 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1148968-9af0-4745-b99a-82a9595fa4c1 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:18.093073 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1462/2917] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:47:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:47:18.387022 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d115b3-1f85-4384-a565-e743236a0f1e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:18.388602 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-02d115b3-1f85-4384-a565-e743236a0f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:18.401195 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d115b3-1f85-4384-a565-e743236a0f1e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:18.406901 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f780691-f42d-4a7f-bf26-697e29c87f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:18.407641 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f780691-f42d-4a7f-bf26-697e29c87f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:18.407931 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f780691-f42d-4a7f-bf26-697e29c87f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:18.408232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f780691-f42d-4a7f-bf26-697e29c87f63 admin admin] PATH_INFO: `/v3/projects/4bcaf3ee57fe4f2091b17cc66d2c1aa0` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:18.409491 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f780691-f42d-4a7f-bf26-697e29c87f63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bcaf3ee57fe4f2091b17cc66d2c1aa0)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:18.411393 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f780691-f42d-4a7f-bf26-697e29c87f63 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:18.461678 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7f780691-f42d-4a7f-bf26-697e29c87f63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bcaf3ee57fe4f2091b17cc66d2c1aa0', 'request_id': 'req-7f780691-f42d-4a7f-bf26-697e29c87f63'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:47:18.471362 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7f780691-f42d-4a7f-bf26-697e29c87f63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bcaf3ee57fe4f2091b17cc66d2c1aa0', 'request_id': 'req-7f780691-f42d-4a7f-bf26-697e29c87f63'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:47:18.477134 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7f780691-f42d-4a7f-bf26-697e29c87f63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bcaf3ee57fe4f2091b17cc66d2c1aa0', 'request_id': 'req-7f780691-f42d-4a7f-bf26-697e29c87f63'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:47:18.488077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7f780691-f42d-4a7f-bf26-697e29c87f63 admin admin] The token cache is being invalidate because project 4bcaf3ee57fe4f2091b17cc66d2c1aa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:47:18.488360 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7f780691-f42d-4a7f-bf26-697e29c87f63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:47:18.488980 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7f780691-f42d-4a7f-bf26-697e29c87f63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:47:18.490501 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1456/2918] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:47:18 2022] DELETE /identity/v3/projects/4bcaf3ee57fe4f2091b17cc66d2c1aa0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:47:18.495827 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc39d0d-36f7-400f-be24-3fc6f44d252f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:18.496110 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc39d0d-36f7-400f-be24-3fc6f44d252f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:18.496385 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc39d0d-36f7-400f-be24-3fc6f44d252f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:18.548048 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bdc39d0d-36f7-400f-be24-3fc6f44d252f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:47:18.549936 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bdc39d0d-36f7-400f-be24-3fc6f44d252f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:18.620862 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1463/2919] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:47:18.630390 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683d959-547d-43f4-9d97-6ebbcf5a6ad9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:18.630618 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683d959-547d-43f4-9d97-6ebbcf5a6ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:18.630881 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683d959-547d-43f4-9d97-6ebbcf5a6ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:18.652342 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c683d959-547d-43f4-9d97-6ebbcf5a6ad9 None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:47:18.653601 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c683d959-547d-43f4-9d97-6ebbcf5a6ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:18.662692 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1457/2920] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:47:18.668941 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc39d0d-36f7-400f-be24-3fc6f44d252f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:18.678871 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc39d0d-36f7-400f-be24-3fc6f44d252f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:18.683787 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7564b-c4fa-43d2-bf62-8618b0f0c459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:18.684384 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7564b-c4fa-43d2-bf62-8618b0f0c459 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:18.684659 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7564b-c4fa-43d2-bf62-8618b0f0c459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:18.684929 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7564b-c4fa-43d2-bf62-8618b0f0c459 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:18.685612 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7564b-c4fa-43d2-bf62-8618b0f0c459 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:47:18.685910 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7564b-c4fa-43d2-bf62-8618b0f0c459 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:18.687502 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7564b-c4fa-43d2-bf62-8618b0f0c459 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:18.699175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1464/2921] 10.222.0.60 () {58 vars in 1153 bytes} [Sat Sep 17 04:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:47:18.703559 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683d959-547d-43f4-9d97-6ebbcf5a6ad9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:18.716938 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683d959-547d-43f4-9d97-6ebbcf5a6ad9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:18.724152 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc95f2-82a6-493b-a4e6-94df09972901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:18.724753 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc95f2-82a6-493b-a4e6-94df09972901 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:18.725029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc95f2-82a6-493b-a4e6-94df09972901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:18.725373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc95f2-82a6-493b-a4e6-94df09972901 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:18.726078 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc95f2-82a6-493b-a4e6-94df09972901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:18.728111 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc95f2-82a6-493b-a4e6-94df09972901 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:18.759911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1458/2922] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:47:18 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:47:18.764285 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7564b-c4fa-43d2-bf62-8618b0f0c459 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:18.775202 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7564b-c4fa-43d2-bf62-8618b0f0c459 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:18.844096 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee81f2e-cefa-40a6-920a-c105cc17bea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:18.844719 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee81f2e-cefa-40a6-920a-c105cc17bea3 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:18.844995 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee81f2e-cefa-40a6-920a-c105cc17bea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:18.845304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee81f2e-cefa-40a6-920a-c105cc17bea3 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:18.845950 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee81f2e-cefa-40a6-920a-c105cc17bea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:18.847912 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee81f2e-cefa-40a6-920a-c105cc17bea3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:18.865389 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:18.872971 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1465/2923] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:47:18 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:47:18.877671 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc95f2-82a6-493b-a4e6-94df09972901 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:18.888642 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc95f2-82a6-493b-a4e6-94df09972901 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:18.893911 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c12e0-4492-49d5-a5a0-4e4c6d465cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:18.894515 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c12e0-4492-49d5-a5a0-4e4c6d465cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:18.894798 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c12e0-4492-49d5-a5a0-4e4c6d465cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:18.895063 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c12e0-4492-49d5-a5a0-4e4c6d465cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:18.895696 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c12e0-4492-49d5-a5a0-4e4c6d465cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:18.897369 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c12e0-4492-49d5-a5a0-4e4c6d465cb2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:18.907787 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1459/2924] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:47:18.912553 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee81f2e-cefa-40a6-920a-c105cc17bea3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:18.926444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee81f2e-cefa-40a6-920a-c105cc17bea3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:18.932361 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343112fc-5b0c-45dc-ae5b-94ec4b678e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:18.932967 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343112fc-5b0c-45dc-ae5b-94ec4b678e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:18.933256 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343112fc-5b0c-45dc-ae5b-94ec4b678e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:18.933522 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343112fc-5b0c-45dc-ae5b-94ec4b678e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:18.934162 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343112fc-5b0c-45dc-ae5b-94ec4b678e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:18.935764 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343112fc-5b0c-45dc-ae5b-94ec4b678e45 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:18.947179 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1466/2925] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:47:18.952144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c12e0-4492-49d5-a5a0-4e4c6d465cb2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:18.963761 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c12e0-4492-49d5-a5a0-4e4c6d465cb2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:18.969319 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cef6cf-3268-449a-818f-c7b14fb96cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:18.969984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cef6cf-3268-449a-818f-c7b14fb96cf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:18.970289 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cef6cf-3268-449a-818f-c7b14fb96cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:18.970567 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cef6cf-3268-449a-818f-c7b14fb96cf6 admin admin] PATH_INFO: `/v3/projects/e2e4abf23c7a495cb117b10045236a1c/users/bbdf6fa7e7f649e3a854544c7e1fe36c/roles/231a681c35614ea3aaf3452a8c167c5c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:18.986659 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cef6cf-3268-449a-818f-c7b14fb96cf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2e4abf23c7a495cb117b10045236a1c, user_id=bbdf6fa7e7f649e3a854544c7e1fe36c, role_id=231a681c35614ea3aaf3452a8c167c5c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:18.989605 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cef6cf-3268-449a-818f-c7b14fb96cf6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:18.991803 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:19.006122 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1460/2926] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:47:18 2022] PUT /identity/v3/projects/e2e4abf23c7a495cb117b10045236a1c/users/bbdf6fa7e7f649e3a854544c7e1fe36c/roles/231a681c35614ea3aaf3452a8c167c5c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:47:19.011227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448f59b-acc6-45f6-8291-14b67c97900f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:19.011505 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448f59b-acc6-45f6-8291-14b67c97900f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:19.011771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448f59b-acc6-45f6-8291-14b67c97900f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:19.033625 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7448f59b-acc6-45f6-8291-14b67c97900f None None] MFA Rules not processed for user `8a7b708ff6c140a3ba833eb00eff7926`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:47:19.034944 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7448f59b-acc6-45f6-8291-14b67c97900f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:19.111494 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1467/2927] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:47:19.125709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cef6cf-3268-449a-818f-c7b14fb96cf6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:19.128622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a1cef6cf-3268-449a-818f-c7b14fb96cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:19.197521 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cef6cf-3268-449a-818f-c7b14fb96cf6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:19.203144 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03282c5-9a5a-4bad-ba3a-09f86d39172f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:19.203973 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03282c5-9a5a-4bad-ba3a-09f86d39172f service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:19.204359 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03282c5-9a5a-4bad-ba3a-09f86d39172f service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:19.204735 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03282c5-9a5a-4bad-ba3a-09f86d39172f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:19.218116 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03282c5-9a5a-4bad-ba3a-09f86d39172f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:19.220208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03282c5-9a5a-4bad-ba3a-09f86d39172f service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:19.233844 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1461/2928] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:47:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:47:23.271396 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfc46a-78a4-429b-a9be-0101d1652659 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:23.271972 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfc46a-78a4-429b-a9be-0101d1652659 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:23.272092 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfc46a-78a4-429b-a9be-0101d1652659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:23.305577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2ebfc46a-78a4-429b-a9be-0101d1652659 None None] MFA Rules not processed for user `bbdf6fa7e7f649e3a854544c7e1fe36c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:47:23.306793 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2ebfc46a-78a4-429b-a9be-0101d1652659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:23.377663 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1468/2929] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:47:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:47:23.382440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03282c5-9a5a-4bad-ba3a-09f86d39172f service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:23.393299 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03282c5-9a5a-4bad-ba3a-09f86d39172f service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:23.398040 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4163c1a-a5a7-4464-904d-e45c7970ea18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:23.398652 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4163c1a-a5a7-4464-904d-e45c7970ea18 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:23.398922 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4163c1a-a5a7-4464-904d-e45c7970ea18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:23.399217 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4163c1a-a5a7-4464-904d-e45c7970ea18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:23.399900 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4163c1a-a5a7-4464-904d-e45c7970ea18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:23.401840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4163c1a-a5a7-4464-904d-e45c7970ea18 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:23.431091 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1462/2930] 10.222.0.60 () {60 vars in 1153 bytes} [Sat Sep 17 04:47:23 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:47:23.435183 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfc46a-78a4-429b-a9be-0101d1652659 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:23.446540 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfc46a-78a4-429b-a9be-0101d1652659 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:23.509737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ce770-8570-42e2-9549-62d2739b1deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:23.510424 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ce770-8570-42e2-9549-62d2739b1deb admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:23.510768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ce770-8570-42e2-9549-62d2739b1deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:23.511101 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ce770-8570-42e2-9549-62d2739b1deb admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:23.511823 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ce770-8570-42e2-9549-62d2739b1deb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:23.513870 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ce770-8570-42e2-9549-62d2739b1deb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:23.549028 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1469/2931] 10.222.0.60 () {60 vars in 1144 bytes} [Sat Sep 17 04:47:23 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:47:23.553202 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4163c1a-a5a7-4464-904d-e45c7970ea18 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:23.563730 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4163c1a-a5a7-4464-904d-e45c7970ea18 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:23.569839 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d28612-9d74-4687-a38c-eade168aee46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:23.570840 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d28612-9d74-4687-a38c-eade168aee46 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:23.571105 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d28612-9d74-4687-a38c-eade168aee46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:23.571828 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d28612-9d74-4687-a38c-eade168aee46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:23.572496 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d28612-9d74-4687-a38c-eade168aee46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:23.574151 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d28612-9d74-4687-a38c-eade168aee46 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:23.591460 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1463/2932] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:47:23.592588 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ce770-8570-42e2-9549-62d2739b1deb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:23.605218 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ce770-8570-42e2-9549-62d2739b1deb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:23.609894 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091908cd-6ce0-472f-a27b-8d60d3688fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:23.610576 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091908cd-6ce0-472f-a27b-8d60d3688fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:23.610869 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091908cd-6ce0-472f-a27b-8d60d3688fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:23.611141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091908cd-6ce0-472f-a27b-8d60d3688fc3 admin admin] PATH_INFO: `/v3/projects/162b40f4ef74498eb336fe4869eaa08b/users/4709e5bf50d94b87858245adf1a57ce3/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:23.613740 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:23.629674 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091908cd-6ce0-472f-a27b-8d60d3688fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=162b40f4ef74498eb336fe4869eaa08b, user_id=4709e5bf50d94b87858245adf1a57ce3, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:23.632316 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091908cd-6ce0-472f-a27b-8d60d3688fc3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:23.647462 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1470/2933] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:47:23 2022] PUT /identity/v3/projects/162b40f4ef74498eb336fe4869eaa08b/users/4709e5bf50d94b87858245adf1a57ce3/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:47:23.652100 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d28612-9d74-4687-a38c-eade168aee46 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:23.662971 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d28612-9d74-4687-a38c-eade168aee46 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:23.720415 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8520e-a268-4aea-8238-260905ed3ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:23.721032 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8520e-a268-4aea-8238-260905ed3ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:23.721317 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8520e-a268-4aea-8238-260905ed3ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:23.721593 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8520e-a268-4aea-8238-260905ed3ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:23.722285 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8520e-a268-4aea-8238-260905ed3ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:23.723909 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8520e-a268-4aea-8238-260905ed3ee5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:23.735150 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1464/2934] 10.222.0.60 () {58 vars in 1122 bytes} [Sat Sep 17 04:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:47:23.739504 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091908cd-6ce0-472f-a27b-8d60d3688fc3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:23.750147 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091908cd-6ce0-472f-a27b-8d60d3688fc3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:23.755711 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be280e-88ff-4bb7-850e-1d9ebc5e4dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:23.756466 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be280e-88ff-4bb7-850e-1d9ebc5e4dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:23.756800 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be280e-88ff-4bb7-850e-1d9ebc5e4dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:23.757195 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be280e-88ff-4bb7-850e-1d9ebc5e4dcb admin admin] PATH_INFO: `/v3/projects/162b40f4ef74498eb336fe4869eaa08b/users/4709e5bf50d94b87858245adf1a57ce3/roles/cbe495373d2646beae215fdffe6f2ce9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:23.759329 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be280e-88ff-4bb7-850e-1d9ebc5e4dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=162b40f4ef74498eb336fe4869eaa08b, user_id=4709e5bf50d94b87858245adf1a57ce3, role_id=cbe495373d2646beae215fdffe6f2ce9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:23.762071 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be280e-88ff-4bb7-850e-1d9ebc5e4dcb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:23.773634 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1471/2935] 10.222.0.60 () {60 vars in 1483 bytes} [Sat Sep 17 04:47:23 2022] PUT /identity/v3/projects/162b40f4ef74498eb336fe4869eaa08b/users/4709e5bf50d94b87858245adf1a57ce3/roles/cbe495373d2646beae215fdffe6f2ce9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:47:28.265834 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76c89a-d62c-46b0-afdd-fbf729819fbd None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:28.266375 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76c89a-d62c-46b0-afdd-fbf729819fbd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:28.266588 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76c89a-d62c-46b0-afdd-fbf729819fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:28.300128 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1b76c89a-d62c-46b0-afdd-fbf729819fbd None None] MFA Rules not processed for user `4709e5bf50d94b87858245adf1a57ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:47:28.301449 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1b76c89a-d62c-46b0-afdd-fbf729819fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:28.373015 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:28.399125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1465/2936] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:47:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:47:28.415386 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be280e-88ff-4bb7-850e-1d9ebc5e4dcb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:28.417660 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-11be280e-88ff-4bb7-850e-1d9ebc5e4dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:28.541197 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be280e-88ff-4bb7-850e-1d9ebc5e4dcb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:28.547134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1e0a6-720d-403e-8313-39cf7cafdb57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:28.547805 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1e0a6-720d-403e-8313-39cf7cafdb57 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:28.548076 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1e0a6-720d-403e-8313-39cf7cafdb57 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:28.548283 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1e0a6-720d-403e-8313-39cf7cafdb57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:28.619282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1e0a6-720d-403e-8313-39cf7cafdb57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:28.620940 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1e0a6-720d-403e-8313-39cf7cafdb57 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:28.630680 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1472/2937] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:47:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:47:29.637391 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7aafc-bb89-4c08-9251-b1b41529342e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:29.637923 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7aafc-bb89-4c08-9251-b1b41529342e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:29.638056 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7aafc-bb89-4c08-9251-b1b41529342e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:29.659940 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d9b7aafc-bb89-4c08-9251-b1b41529342e None None] MFA Rules not processed for user `bbdf6fa7e7f649e3a854544c7e1fe36c`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:47:29.661208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d9b7aafc-bb89-4c08-9251-b1b41529342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:29.669138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1466/2938] 10.222.0.60 () {58 vars in 958 bytes} [Sat Sep 17 04:47:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:47:38.626332 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=80,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:39.653709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=10 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:47.564599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1e0a6-720d-403e-8313-39cf7cafdb57 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:47.566282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6fb1e0a6-720d-403e-8313-39cf7cafdb57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:47.661306 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1e0a6-720d-403e-8313-39cf7cafdb57 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:47.666993 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6de1f4-3e8a-4660-95d3-452737fe22be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:47.667595 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6de1f4-3e8a-4660-95d3-452737fe22be service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:47.667835 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6de1f4-3e8a-4660-95d3-452737fe22be service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:47.668134 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6de1f4-3e8a-4660-95d3-452737fe22be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:47.670126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8e6de1f4-3e8a-4660-95d3-452737fe22be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:47.682667 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6de1f4-3e8a-4660-95d3-452737fe22be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:47.684237 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6de1f4-3e8a-4660-95d3-452737fe22be service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:47.694141 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1473/2939] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:47:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:47:49.557663 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7aafc-bb89-4c08-9251-b1b41529342e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:49.559307 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d9b7aafc-bb89-4c08-9251-b1b41529342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:49.618489 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7aafc-bb89-4c08-9251-b1b41529342e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:49.622472 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5253acc-c741-46b3-9386-a4e3091e3a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:49.623077 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5253acc-c741-46b3-9386-a4e3091e3a9d service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:49.623355 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5253acc-c741-46b3-9386-a4e3091e3a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:49.623622 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5253acc-c741-46b3-9386-a4e3091e3a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:49.625548 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d5253acc-c741-46b3-9386-a4e3091e3a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:49.636426 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5253acc-c741-46b3-9386-a4e3091e3a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:49.638189 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5253acc-c741-46b3-9386-a4e3091e3a9d service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:49.648143 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1467/2940] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:47:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:47:51.133115 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6de1f4-3e8a-4660-95d3-452737fe22be service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:51.144206 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6de1f4-3e8a-4660-95d3-452737fe22be service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:51.150340 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab51bf3-a27f-4c1f-bbcc-197b214f4f6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:51.150945 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab51bf3-a27f-4c1f-bbcc-197b214f4f6e service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:51.151217 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab51bf3-a27f-4c1f-bbcc-197b214f4f6e service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:51.151480 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab51bf3-a27f-4c1f-bbcc-197b214f4f6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:51.153290 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0ab51bf3-a27f-4c1f-bbcc-197b214f4f6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:51.165062 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab51bf3-a27f-4c1f-bbcc-197b214f4f6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:51.166834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab51bf3-a27f-4c1f-bbcc-197b214f4f6e service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:51.177201 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1474/2941] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:47:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:47:53.730464 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5253acc-c741-46b3-9386-a4e3091e3a9d service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:47:53.732125 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d5253acc-c741-46b3-9386-a4e3091e3a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:53.749499 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5253acc-c741-46b3-9386-a4e3091e3a9d service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:47:53.755411 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e625d3-24f1-4911-854f-b3c7ff562546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:47:53.757055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e625d3-24f1-4911-854f-b3c7ff562546 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:47:53.757055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e625d3-24f1-4911-854f-b3c7ff562546 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:47:53.757055 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e625d3-24f1-4911-854f-b3c7ff562546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:47:53.758370 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c1e625d3-24f1-4911-854f-b3c7ff562546 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:47:53.771414 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e625d3-24f1-4911-854f-b3c7ff562546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:47:53.773212 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e625d3-24f1-4911-854f-b3c7ff562546 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:47:53.785468 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1468/2942] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:47:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:48:01.177829 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=47 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:48:03.780232 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=32 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:48:06.747670 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab51bf3-a27f-4c1f-bbcc-197b214f4f6e service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:48:06.764403 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab51bf3-a27f-4c1f-bbcc-197b214f4f6e service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:48:06.770311 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0790cc-7020-4a7a-be1e-0d186b1c2034 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:48:06.770948 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0790cc-7020-4a7a-be1e-0d186b1c2034 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:48:06.771210 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0790cc-7020-4a7a-be1e-0d186b1c2034 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:48:06.771472 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0790cc-7020-4a7a-be1e-0d186b1c2034 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:48:06.773604 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ba0790cc-7020-4a7a-be1e-0d186b1c2034 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:48:06.788779 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0790cc-7020-4a7a-be1e-0d186b1c2034 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:48:06.790763 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0790cc-7020-4a7a-be1e-0d186b1c2034 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:48:06.803271 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1475/2943] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:48:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:48:09.475843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e625d3-24f1-4911-854f-b3c7ff562546 service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:48:09.486138 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e625d3-24f1-4911-854f-b3c7ff562546 service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:48:09.490685 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b767edc-778d-4231-842c-114ff17fae3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:48:09.491265 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b767edc-778d-4231-842c-114ff17fae3b service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:48:09.491541 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b767edc-778d-4231-842c-114ff17fae3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:48:09.491803 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b767edc-778d-4231-842c-114ff17fae3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:48:09.501570 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b767edc-778d-4231-842c-114ff17fae3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:48:09.502996 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b767edc-778d-4231-842c-114ff17fae3b service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:48:09.512607 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1469/2944] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:48:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:48:11.153204 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0790cc-7020-4a7a-be1e-0d186b1c2034 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:48:11.170582 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0790cc-7020-4a7a-be1e-0d186b1c2034 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:48:11.177243 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7ebdb-8f23-44ea-816f-23a050416fdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3272d57d5d24e5a990a6d7c86e6d4bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:48:11.177885 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7ebdb-8f23-44ea-816f-23a050416fdd service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:48:11.178159 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7ebdb-8f23-44ea-816f-23a050416fdd service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:48:11.178425 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7ebdb-8f23-44ea-816f-23a050416fdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:48:11.180275 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-93b7ebdb-8f23-44ea-816f-23a050416fdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:48:11.194599 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7ebdb-8f23-44ea-816f-23a050416fdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:48:11.196188 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7ebdb-8f23-44ea-816f-23a050416fdd service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:48:11.208581 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1476/2945] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:48:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:48:19.508422 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=6 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:48:21.203186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=12 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:49:44.287097 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b767edc-778d-4231-842c-114ff17fae3b service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:49:44.296778 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b767edc-778d-4231-842c-114ff17fae3b service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:49:44.301402 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692ca0a-55f8-497a-9177-aae835633f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:49:44.301998 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692ca0a-55f8-497a-9177-aae835633f26 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:49:44.302274 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692ca0a-55f8-497a-9177-aae835633f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:49:44.302545 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692ca0a-55f8-497a-9177-aae835633f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:49:44.304697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6692ca0a-55f8-497a-9177-aae835633f26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:49:44.315290 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692ca0a-55f8-497a-9177-aae835633f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:49:44.316715 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692ca0a-55f8-497a-9177-aae835633f26 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:49:44.325204 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1470/2946] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:49:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:49:54.321682 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=6 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:50:25.287913 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7ebdb-8f23-44ea-816f-23a050416fdd service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:50:25.298288 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7ebdb-8f23-44ea-816f-23a050416fdd service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:50:25.302577 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ab1d5-3aad-461a-a1b7-5dbfd18a5518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:50:25.303190 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ab1d5-3aad-461a-a1b7-5dbfd18a5518 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:50:25.303464 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ab1d5-3aad-461a-a1b7-5dbfd18a5518 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:50:25.303727 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ab1d5-3aad-461a-a1b7-5dbfd18a5518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:50:25.305702 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6b0ab1d5-3aad-461a-a1b7-5dbfd18a5518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:50:25.319644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ab1d5-3aad-461a-a1b7-5dbfd18a5518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:50:25.321072 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ab1d5-3aad-461a-a1b7-5dbfd18a5518 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:50:25.332689 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1477/2947] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:50:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:50:26.533456 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692ca0a-55f8-497a-9177-aae835633f26 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:50:26.542808 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692ca0a-55f8-497a-9177-aae835633f26 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:50:26.546826 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e2bdf-ce33-4c83-8659-bbb297141420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:50:26.547433 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e2bdf-ce33-4c83-8659-bbb297141420 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:50:26.547709 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e2bdf-ce33-4c83-8659-bbb297141420 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:50:26.547978 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e2bdf-ce33-4c83-8659-bbb297141420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:50:26.549781 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d60e2bdf-ce33-4c83-8659-bbb297141420 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:50:26.564532 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e2bdf-ce33-4c83-8659-bbb297141420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:50:26.566149 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e2bdf-ce33-4c83-8659-bbb297141420 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:50:26.577585 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1471/2948] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:50:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:50:35.327282 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:50:36.572582 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=6 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:50:43.508175 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ab1d5-3aad-461a-a1b7-5dbfd18a5518 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:50:43.521693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ab1d5-3aad-461a-a1b7-5dbfd18a5518 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:50:43.528994 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bc0b6-d298-4c39-a9ff-3ea2e4220381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:50:43.529878 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bc0b6-d298-4c39-a9ff-3ea2e4220381 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:50:43.530297 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bc0b6-d298-4c39-a9ff-3ea2e4220381 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:50:43.530639 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bc0b6-d298-4c39-a9ff-3ea2e4220381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:50:43.533771 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f99bc0b6-d298-4c39-a9ff-3ea2e4220381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:50:43.544903 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99bc0b6-d298-4c39-a9ff-3ea2e4220381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:50:43.546361 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99bc0b6-d298-4c39-a9ff-3ea2e4220381 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:50:43.554933 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1478/2949] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:50:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:50:43.677697 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e2bdf-ce33-4c83-8659-bbb297141420 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:50:43.686551 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e2bdf-ce33-4c83-8659-bbb297141420 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:50:43.690653 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e16878-48c2-4b6e-9a77-3057a2c0f1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:50:43.691267 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e16878-48c2-4b6e-9a77-3057a2c0f1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:50:43.691539 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e16878-48c2-4b6e-9a77-3057a2c0f1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:50:43.691856 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e16878-48c2-4b6e-9a77-3057a2c0f1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:50:43.693984 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e1e16878-48c2-4b6e-9a77-3057a2c0f1b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:50:43.705110 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e16878-48c2-4b6e-9a77-3057a2c0f1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:50:43.706536 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e16878-48c2-4b6e-9a77-3057a2c0f1b8 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:50:43.715047 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1472/2950] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:50:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:50:43.837021 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bc0b6-d298-4c39-a9ff-3ea2e4220381 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:50:43.846335 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bc0b6-d298-4c39-a9ff-3ea2e4220381 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:50:43.850564 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40ed8c-c8ca-45df-bd02-52f4732a8f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:50:43.851186 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40ed8c-c8ca-45df-bd02-52f4732a8f4e service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:50:43.851457 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40ed8c-c8ca-45df-bd02-52f4732a8f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:50:43.851726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40ed8c-c8ca-45df-bd02-52f4732a8f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:50:43.853644 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-da40ed8c-c8ca-45df-bd02-52f4732a8f4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:50:43.867866 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40ed8c-c8ca-45df-bd02-52f4732a8f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:50:43.869370 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40ed8c-c8ca-45df-bd02-52f4732a8f4e service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:50:43.880568 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1479/2951] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:50:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:50:53.711268 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=6 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:50:53.875737 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=12 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:52:28.664892 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e16878-48c2-4b6e-9a77-3057a2c0f1b8 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:52:28.675033 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e16878-48c2-4b6e-9a77-3057a2c0f1b8 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:52:28.679665 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e6575-dcbe-4b84-9e2c-ef5c90d92e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:52:28.680272 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e6575-dcbe-4b84-9e2c-ef5c90d92e96 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:52:28.680549 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e6575-dcbe-4b84-9e2c-ef5c90d92e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:52:28.680813 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e6575-dcbe-4b84-9e2c-ef5c90d92e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:52:28.689612 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e6575-dcbe-4b84-9e2c-ef5c90d92e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:52:28.691029 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e6575-dcbe-4b84-9e2c-ef5c90d92e96 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:52:28.701121 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1473/2952] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:52:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:52:34.740209 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40ed8c-c8ca-45df-bd02-52f4732a8f4e service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:52:34.749623 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40ed8c-c8ca-45df-bd02-52f4732a8f4e service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:52:34.753590 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98447d3b-74a9-43e0-8337-14b9a21db176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9360be7ee9e4739b20c59955d28da47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:52:34.754278 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98447d3b-74a9-43e0-8337-14b9a21db176 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:52:34.754560 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98447d3b-74a9-43e0-8337-14b9a21db176 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:52:34.754833 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98447d3b-74a9-43e0-8337-14b9a21db176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:52:34.818010 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98447d3b-74a9-43e0-8337-14b9a21db176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:52:34.819449 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98447d3b-74a9-43e0-8337-14b9a21db176 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:52:34.829645 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1480/2953] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:52:37.957620 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e6575-dcbe-4b84-9e2c-ef5c90d92e96 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:52:37.959239 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ce0e6575-dcbe-4b84-9e2c-ef5c90d92e96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:52:37.980453 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e6575-dcbe-4b84-9e2c-ef5c90d92e96 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:52:37.986215 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fb442-f686-4f08-908b-c578e126483f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:52:37.986915 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fb442-f686-4f08-908b-c578e126483f service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:52:37.987260 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fb442-f686-4f08-908b-c578e126483f service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:52:37.987594 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fb442-f686-4f08-908b-c578e126483f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:52:37.989712 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-899fb442-f686-4f08-908b-c578e126483f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:52:38.003471 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fb442-f686-4f08-908b-c578e126483f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:52:38.005065 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fb442-f686-4f08-908b-c578e126483f service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:52:38.018295 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1474/2954] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:52:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:52:40.895317 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98447d3b-74a9-43e0-8337-14b9a21db176 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:52:40.914552 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98447d3b-74a9-43e0-8337-14b9a21db176 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:52:40.920224 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8df207-dd59-4e07-825d-ec22eb22621d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:52:40.921334 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8df207-dd59-4e07-825d-ec22eb22621d service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:52:40.921469 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8df207-dd59-4e07-825d-ec22eb22621d service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:52:40.921660 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8df207-dd59-4e07-825d-ec22eb22621d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:52:40.923432 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fe8df207-dd59-4e07-825d-ec22eb22621d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:52:40.937099 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8df207-dd59-4e07-825d-ec22eb22621d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:52:40.938753 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8df207-dd59-4e07-825d-ec22eb22621d service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:52:40.956373 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fb442-f686-4f08-908b-c578e126483f service placement] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:52:40.957269 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1481/2955] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:52:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:52:40.979208 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fb442-f686-4f08-908b-c578e126483f service placement] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:52:40.984023 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:52:40.984675 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:52:40.984956 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:52:40.985302 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] PATH_INFO: `/v3/users/bbdf6fa7e7f649e3a854544c7e1fe36c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:52:40.989045 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbdf6fa7e7f649e3a854544c7e1fe36c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:52:40.991064 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:52:41.042986 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbdf6fa7e7f649e3a854544c7e1fe36c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.051760 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbdf6fa7e7f649e3a854544c7e1fe36c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.052440 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbdf6fa7e7f649e3a854544c7e1fe36c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.060755 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbdf6fa7e7f649e3a854544c7e1fe36c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.073298 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbdf6fa7e7f649e3a854544c7e1fe36c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.080132 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1475/2956] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:52:40 2022] DELETE /identity/v3/users/bbdf6fa7e7f649e3a854544c7e1fe36c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:52:41.270709 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8df207-dd59-4e07-825d-ec22eb22621d service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:52:41.281627 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8df207-dd59-4e07-825d-ec22eb22621d service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:52:41.340515 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:52:41.341112 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:52:41.341393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:52:41.341666 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] PATH_INFO: `/v3/projects/e2e4abf23c7a495cb117b10045236a1c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:52:41.342826 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2e4abf23c7a495cb117b10045236a1c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:52:41.344726 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:52:41.392440 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2e4abf23c7a495cb117b10045236a1c', 'request_id': 'req-5dca37e6-643f-4952-b1c5-ce807da9b744'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.400548 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2e4abf23c7a495cb117b10045236a1c', 'request_id': 'req-5dca37e6-643f-4952-b1c5-ce807da9b744'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.406420 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2e4abf23c7a495cb117b10045236a1c', 'request_id': 'req-5dca37e6-643f-4952-b1c5-ce807da9b744'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.417372 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] The token cache is being invalidate because project e2e4abf23c7a495cb117b10045236a1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:52:41.417760 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.418394 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.419899 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1482/2957] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:52:41 2022] DELETE /identity/v3/projects/e2e4abf23c7a495cb117b10045236a1c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:52:41.424171 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:52:41.425799 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:52:41.439503 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb42d4a-6675-4df7-b96c-f4f84611bde2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:52:41.499081 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:52:41.499726 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:52:41.500026 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:52:41.500281 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] PATH_INFO: `/v3/users/4709e5bf50d94b87858245adf1a57ce3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:52:41.501587 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4709e5bf50d94b87858245adf1a57ce3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:52:41.503557 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:52:41.552783 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4709e5bf50d94b87858245adf1a57ce3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.554599 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:52:41.561843 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4709e5bf50d94b87858245adf1a57ce3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.562644 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4709e5bf50d94b87858245adf1a57ce3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.570952 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4709e5bf50d94b87858245adf1a57ce3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.582887 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4709e5bf50d94b87858245adf1a57ce3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.588934 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1476/2958] 10.222.0.60 () {58 vars in 1224 bytes} [Sat Sep 17 04:52:41 2022] DELETE /identity/v3/users/4709e5bf50d94b87858245adf1a57ce3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:52:41.773304 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:52:41.774927 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:52:41.788121 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca37e6-643f-4952-b1c5-ce807da9b744 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:52:41.797227 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:52:41.849834 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a7b708ff6c140a3ba833eb00eff7926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fd2fb11102f467d836ef30d6efdd922', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:52:41.850463 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:52:41.850768 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:52:41.851035 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] PATH_INFO: `/v3/projects/162b40f4ef74498eb336fe4869eaa08b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:52:41.852161 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=162b40f4ef74498eb336fe4869eaa08b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:52:41.854131 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:52:41.901932 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '162b40f4ef74498eb336fe4869eaa08b', 'request_id': 'req-076762ef-2773-4e2d-92e2-a408188faf4e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.911061 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '162b40f4ef74498eb336fe4869eaa08b', 'request_id': 'req-076762ef-2773-4e2d-92e2-a408188faf4e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.917019 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '162b40f4ef74498eb336fe4869eaa08b', 'request_id': 'req-076762ef-2773-4e2d-92e2-a408188faf4e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.928299 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] The token cache is being invalidate because project 162b40f4ef74498eb336fe4869eaa08b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:52:41.928550 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.929214 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:52:41.930723 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1483/2959] 10.222.0.60 () {58 vars in 1233 bytes} [Sat Sep 17 04:52:41 2022] DELETE /identity/v3/projects/162b40f4ef74498eb336fe4869eaa08b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:52:51.585720 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=12 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:52:51.925444 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:52:53.565752 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:52:53.567331 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:52:53.644404 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a548c-34d3-47e8-a720-55cdb9af8ec9 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:52:53.705870 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9179c8b-4adf-496d-bbaa-6d2bcd6e5fc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:52:53.706493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9179c8b-4adf-496d-bbaa-6d2bcd6e5fc3 service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:52:53.706770 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9179c8b-4adf-496d-bbaa-6d2bcd6e5fc3 service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:52:53.707043 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9179c8b-4adf-496d-bbaa-6d2bcd6e5fc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:52:53.708854 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d9179c8b-4adf-496d-bbaa-6d2bcd6e5fc3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:52:53.720493 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9179c8b-4adf-496d-bbaa-6d2bcd6e5fc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:52:53.722133 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9179c8b-4adf-496d-bbaa-6d2bcd6e5fc3 service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:52:53.731988 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1477/2960] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:52:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:53:03.727779 dv-cinder-857993-2-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=58 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:58:00.556126 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:58:00.557693 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:58:00.648602 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076762ef-2773-4e2d-92e2-a408188faf4e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:58:00.654252 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868d4cc-7955-4c57-8de9-d2af02e1aba2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d3b7887c13466ab8e3d9737f907c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58466195aca94b188d82c1b3a647e767', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:58:00.654859 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868d4cc-7955-4c57-8de9-d2af02e1aba2 service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:58:00.655145 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868d4cc-7955-4c57-8de9-d2af02e1aba2 service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:58:00.655393 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868d4cc-7955-4c57-8de9-d2af02e1aba2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:58:00.664761 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868d4cc-7955-4c57-8de9-d2af02e1aba2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:58:00.666428 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868d4cc-7955-4c57-8de9-d2af02e1aba2 service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:58:00.676606 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1484/2961] 10.222.0.60 () {60 vars in 1365 bytes} [Sat Sep 17 04:58:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:58:10.672280 dv-cinder-857993-2-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=37 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}